Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_single_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_single_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 3,
 'device': device(type='cuda', index=1),
 'dropout': 0.05,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_single/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1400,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1400,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_single_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_single/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_single/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.545305
Epoch 1
Validation auc = 0.526291
Epoch 2
Validation auc = 0.718075
Epoch 3
Validation auc = 0.600704
Epoch 4
Loss = 1.8771e+01, PNorm = 68.3704, GNorm = 249.4009, lr_0 = 6.5000e-04
Validation auc = 0.710798
Epoch 5
Validation auc = 0.700235
Epoch 6
Validation auc = 0.715962
Epoch 7
Validation auc = 0.657746
Epoch 8
Validation auc = 0.700000
Epoch 9
Loss = 1.3844e+01, PNorm = 68.5282, GNorm = 846.8184, lr_0 = 9.9743e-04
Validation auc = 0.554225
Epoch 10
Validation auc = 0.595775
Epoch 11
Validation auc = 0.570188
Epoch 12
Validation auc = 0.608685
Epoch 13
Validation auc = 0.586385
Epoch 14
Loss = 3.2477e+01, PNorm = 68.6862, GNorm = 2068.0768, lr_0 = 9.8890e-04
Validation auc = 0.618075
Epoch 15
Validation auc = 0.628873
Epoch 16
Validation auc = 0.669484
Epoch 17
Validation auc = 0.665728
Epoch 18
Validation auc = 0.729812
Epoch 19
Loss = 1.6275e+01, PNorm = 68.8136, GNorm = 1501.5605, lr_0 = 9.8045e-04
Validation auc = 0.747887
Epoch 20
Validation auc = 0.615258
Epoch 21
Validation auc = 0.653991
Epoch 22
Validation auc = 0.642488
Epoch 23
Validation auc = 0.662441
Epoch 24
Loss = 9.6419e+00, PNorm = 68.9231, GNorm = 335.2321, lr_0 = 9.7207e-04
Validation auc = 0.688732
Epoch 25
Validation auc = 0.688732
Epoch 26
Validation auc = 0.689202
Epoch 27
Validation auc = 0.693427
Epoch 28
Validation auc = 0.695305
Epoch 29
Loss = 1.2771e+01, PNorm = 69.0248, GNorm = 2483.8465, lr_0 = 9.6376e-04
Validation auc = 0.677700
Epoch 30
Validation auc = 0.662441
Epoch 31
Validation auc = 0.689437
Epoch 32
Validation auc = 0.676291
Epoch 33
Validation auc = 0.709624
Epoch 34
Loss = 1.9774e+01, PNorm = 69.1533, GNorm = 2335.2785, lr_0 = 9.5552e-04
Validation auc = 0.735211
Epoch 35
Validation auc = 0.669014
Epoch 36
Validation auc = 0.678873
Epoch 37
Validation auc = 0.718779
Epoch 38
Validation auc = 0.716432
Epoch 39
Loss = 1.1333e+01, PNorm = 69.3184, GNorm = 1511.5677, lr_0 = 9.4735e-04
Validation auc = 0.673944
Epoch 40
Validation auc = 0.682864
Epoch 41
Validation auc = 0.678638
Epoch 42
Validation auc = 0.637324
Epoch 43
Validation auc = 0.645540
Epoch 44
Loss = 1.1331e+01, PNorm = 69.4894, GNorm = 378.1337, lr_0 = 9.3925e-04
Validation auc = 0.649765
Epoch 45
Validation auc = 0.667606
Epoch 46
Validation auc = 0.681690
Epoch 47
Validation auc = 0.615962
Epoch 48
Validation auc = 0.591315
Epoch 49
Loss = 1.9913e+01, PNorm = 69.7122, GNorm = 1479.2073, lr_0 = 9.3122e-04
Validation auc = 0.646009
Epoch 50
Validation auc = 0.623005
Epoch 51
Validation auc = 0.551643
Epoch 52
Validation auc = 0.638263
Epoch 53
Validation auc = 0.683099
Epoch 54
Loss = 1.1752e+01, PNorm = 70.0722, GNorm = 1793.1802, lr_0 = 9.2326e-04
Validation auc = 0.710329
Epoch 55
Validation auc = 0.672300
Epoch 56
Validation auc = 0.705869
Epoch 57
Validation auc = 0.719249
Epoch 58
Validation auc = 0.712441
Epoch 59
Loss = 8.0203e+00, PNorm = 70.4511, GNorm = 1504.4795, lr_0 = 9.1537e-04
Validation auc = 0.721596
Epoch 60
Validation auc = 0.728404
Epoch 61
Validation auc = 0.682629
Epoch 62
Validation auc = 0.685915
Epoch 63
Validation auc = 0.721362
Epoch 64
Loss = 9.5764e+00, PNorm = 70.8591, GNorm = 1391.4417, lr_0 = 9.0754e-04
Validation auc = 0.715493
Epoch 65
Validation auc = 0.720657
Epoch 66
Validation auc = 0.719953
Epoch 67
Validation auc = 0.702347
Epoch 68
Validation auc = 0.692723
Epoch 69
Loss = 4.4400e+00, PNorm = 71.3813, GNorm = 139.4782, lr_0 = 8.9978e-04
Validation auc = 0.667840
Epoch 70
Validation auc = 0.653521
Epoch 71
Validation auc = 0.694601
Epoch 72
Validation auc = 0.687089
Epoch 73
Validation auc = 0.652582
Epoch 74
Loss = 7.9534e+00, PNorm = 71.8912, GNorm = 545.4838, lr_0 = 8.9209e-04
Validation auc = 0.637324
Epoch 75
Validation auc = 0.670423
Epoch 76
Validation auc = 0.680751
Epoch 77
Validation auc = 0.687324
Epoch 78
Validation auc = 0.690376
Epoch 79
Loss = 4.6735e+00, PNorm = 72.2558, GNorm = 306.9435, lr_0 = 8.8447e-04
Validation auc = 0.679343
Epoch 80
Validation auc = 0.683333
Epoch 81
Validation auc = 0.686385
Epoch 82
Validation auc = 0.655869
Epoch 83
Validation auc = 0.683333
Epoch 84
Loss = 3.1383e+00, PNorm = 72.6205, GNorm = 124.2142, lr_0 = 8.7691e-04
Validation auc = 0.684038
Epoch 85
Validation auc = 0.690610
Epoch 86
Validation auc = 0.691549
Epoch 87
Validation auc = 0.691549
Epoch 88
Validation auc = 0.692019
Epoch 89
Loss = 4.4977e+00, PNorm = 72.9993, GNorm = 705.6538, lr_0 = 8.6941e-04
Validation auc = 0.689671
Epoch 90
Validation auc = 0.645070
Epoch 91
Validation auc = 0.677230
Epoch 92
Validation auc = 0.706573
Epoch 93
Validation auc = 0.623005
Epoch 94
Loss = 8.8837e+00, PNorm = 73.2373, GNorm = 780.3724, lr_0 = 8.6198e-04
Validation auc = 0.699531
Epoch 95
Validation auc = 0.709859
Epoch 96
Validation auc = 0.655399
Epoch 97
Validation auc = 0.711502
Epoch 98
Validation auc = 0.721596
Epoch 99
Loss = 6.7488e+00, PNorm = 73.5732, GNorm = 909.7160, lr_0 = 8.5461e-04
Validation auc = 0.692488
Epoch 100
Validation auc = 0.743192
Epoch 101
Validation auc = 0.739906
Epoch 102
Validation auc = 0.660563
Epoch 103
Validation auc = 0.724413
Epoch 104
Loss = 8.1343e+00, PNorm = 73.9413, GNorm = 312.5223, lr_0 = 8.4730e-04
Validation auc = 0.734742
Epoch 105
Validation auc = 0.719014
Epoch 106
Validation auc = 0.749296
Epoch 107
Validation auc = 0.735211
Epoch 108
Validation auc = 0.731455
Epoch 109
Loss = 3.5831e+00, PNorm = 74.3997, GNorm = 394.3579, lr_0 = 8.4006e-04
Validation auc = 0.730047
Epoch 110
Validation auc = 0.714554
Epoch 111
Validation auc = 0.730047
Epoch 112
Validation auc = 0.720657
Epoch 113
Validation auc = 0.689437
Epoch 114
Loss = 5.7042e+00, PNorm = 74.6241, GNorm = 491.1629, lr_0 = 8.3288e-04
Validation auc = 0.720657
Epoch 115
Validation auc = 0.724413
Epoch 116
Validation auc = 0.706573
Epoch 117
Validation auc = 0.722066
Epoch 118
Validation auc = 0.727230
Epoch 119
Loss = 2.6575e+00, PNorm = 74.9520, GNorm = 189.5735, lr_0 = 8.2576e-04
Validation auc = 0.689437
Epoch 120
Validation auc = 0.715962
Epoch 121
Validation auc = 0.722066
Epoch 122
Validation auc = 0.726526
Epoch 123
Validation auc = 0.696244
Epoch 124
Loss = 5.2924e+00, PNorm = 75.3909, GNorm = 407.7709, lr_0 = 8.1870e-04
Validation auc = 0.718545
Epoch 125
Validation auc = 0.707277
Epoch 126
Validation auc = 0.731221
Epoch 127
Validation auc = 0.750469
Epoch 128
Validation auc = 0.747887
Epoch 129
Loss = 2.6008e+00, PNorm = 75.6504, GNorm = 168.8610, lr_0 = 8.1170e-04
Validation auc = 0.731455
Epoch 130
Validation auc = 0.733333
Epoch 131
Validation auc = 0.739671
Epoch 132
Validation auc = 0.728873
Epoch 133
Validation auc = 0.693427
Epoch 134
Loss = 5.0641e+00, PNorm = 75.8517, GNorm = 728.1626, lr_0 = 8.0476e-04
Validation auc = 0.718310
Epoch 135
Validation auc = 0.687324
Epoch 136
Validation auc = 0.684272
Epoch 137
Validation auc = 0.707981
Epoch 138
Validation auc = 0.676995
Epoch 139
Loss = 2.0159e+00, PNorm = 76.0748, GNorm = 127.9897, lr_0 = 7.9788e-04
Validation auc = 0.663850
Epoch 140
Validation auc = 0.732629
Epoch 141
Validation auc = 0.759624
Epoch 142
Validation auc = 0.655869
Epoch 143
Validation auc = 0.712207
Epoch 144
Loss = 2.4087e+00, PNorm = 76.2835, GNorm = 111.0274, lr_0 = 7.9106e-04
Validation auc = 0.695775
Epoch 145
Validation auc = 0.665728
Epoch 146
Validation auc = 0.661972
Epoch 147
Validation auc = 0.706573
Epoch 148
Validation auc = 0.727230
Epoch 149
Loss = 5.1404e+00, PNorm = 76.4680, GNorm = 325.0807, lr_0 = 7.8430e-04
Validation auc = 0.664319
Epoch 150
Validation auc = 0.661033
Epoch 151
Validation auc = 0.688732
Epoch 152
Validation auc = 0.708216
Epoch 153
Validation auc = 0.752817
Epoch 154
Loss = 5.5755e+00, PNorm = 76.7924, GNorm = 582.8088, lr_0 = 7.7759e-04
Validation auc = 0.692019
Epoch 155
Validation auc = 0.761033
Epoch 156
Validation auc = 0.744601
Epoch 157
Validation auc = 0.759155
Epoch 158
Validation auc = 0.681690
Epoch 159
Loss = 3.9864e+00, PNorm = 77.2731, GNorm = 323.9106, lr_0 = 7.7095e-04
Validation auc = 0.634507
Epoch 160
Validation auc = 0.725352
Epoch 161
Validation auc = 0.682394
Epoch 162
Validation auc = 0.710798
Epoch 163
Validation auc = 0.768075
Epoch 164
Loss = 6.5362e+00, PNorm = 77.9007, GNorm = 374.2713, lr_0 = 7.6436e-04
Validation auc = 0.694836
Epoch 165
Validation auc = 0.746714
Epoch 166
Validation auc = 0.674178
Epoch 167
Validation auc = 0.689671
Epoch 168
Validation auc = 0.647653
Epoch 169
Loss = 5.0522e+00, PNorm = 78.4261, GNorm = 353.3713, lr_0 = 7.5782e-04
Validation auc = 0.713615
Epoch 170
Validation auc = 0.674178
Epoch 171
Validation auc = 0.704930
Epoch 172
Validation auc = 0.692019
Epoch 173
Validation auc = 0.701878
Epoch 174
Loss = 9.5324e+00, PNorm = 78.8207, GNorm = 407.7555, lr_0 = 7.5134e-04
Validation auc = 0.693427
Epoch 175
Validation auc = 0.705399
Epoch 176
Validation auc = 0.700235
Epoch 177
Validation auc = 0.711972
Epoch 178
Validation auc = 0.712911
Epoch 179
Loss = 5.2551e+00, PNorm = 79.2215, GNorm = 237.3639, lr_0 = 7.4492e-04
Validation auc = 0.678404
Epoch 180
Validation auc = 0.734507
Epoch 181
Validation auc = 0.714319
Epoch 182
Validation auc = 0.658685
Epoch 183
Validation auc = 0.719718
Epoch 184
Loss = 5.2567e+00, PNorm = 79.6166, GNorm = 582.1144, lr_0 = 7.3855e-04
Validation auc = 0.690610
Epoch 185
Validation auc = 0.719718
Epoch 186
Validation auc = 0.653521
Epoch 187
Validation auc = 0.740376
Epoch 188
Validation auc = 0.708216
Epoch 189
Loss = 6.7459e+00, PNorm = 80.1084, GNorm = 1177.0453, lr_0 = 7.3224e-04
Validation auc = 0.750469
Epoch 190
Validation auc = 0.661502
Epoch 191
Validation auc = 0.762207
Epoch 192
Validation auc = 0.725587
Epoch 193
Validation auc = 0.735211
Epoch 194
Loss = 6.8023e+00, PNorm = 80.6438, GNorm = 1217.8759, lr_0 = 7.2598e-04
Validation auc = 0.671362
Epoch 195
Validation auc = 0.714085
Epoch 196
Validation auc = 0.734742
Epoch 197
Validation auc = 0.627700
Epoch 198
Validation auc = 0.726291
Epoch 199
Loss = 2.5649e+00, PNorm = 81.0197, GNorm = 268.9617, lr_0 = 7.1977e-04
Validation auc = 0.717371
Epoch 200
Validation auc = 0.742254
Epoch 201
Validation auc = 0.657512
Epoch 202
Validation auc = 0.727230
Epoch 203
Validation auc = 0.738732
Epoch 204
Loss = 8.1204e-01, PNorm = 81.3807, GNorm = 311.3574, lr_0 = 7.1362e-04
Validation auc = 0.739437
Epoch 205
Validation auc = 0.666901
Epoch 206
Validation auc = 0.650000
Epoch 207
Validation auc = 0.724413
Epoch 208
Validation auc = 0.729812
Epoch 209
Loss = 1.7349e+00, PNorm = 81.6869, GNorm = 75.3167, lr_0 = 7.0752e-04
Validation auc = 0.676526
Epoch 210
Validation auc = 0.669484
Epoch 211
Validation auc = 0.731455
Epoch 212
Validation auc = 0.753756
Epoch 213
Validation auc = 0.629343
Epoch 214
Loss = 6.5698e+00, PNorm = 82.0240, GNorm = 1047.0602, lr_0 = 7.0147e-04
Validation auc = 0.686385
Epoch 215
Validation auc = 0.706103
Epoch 216
Validation auc = 0.701878
Epoch 217
Validation auc = 0.648357
Epoch 218
Validation auc = 0.658451
Epoch 219
Loss = 3.1900e+00, PNorm = 82.3414, GNorm = 164.2188, lr_0 = 6.9548e-04
Validation auc = 0.720188
Epoch 220
Validation auc = 0.710563
Epoch 221
Validation auc = 0.676995
Epoch 222
Validation auc = 0.652113
Epoch 223
Validation auc = 0.711502
Epoch 224
Loss = 5.5989e+00, PNorm = 82.8049, GNorm = 500.6037, lr_0 = 6.8953e-04
Validation auc = 0.714319
Epoch 225
Validation auc = 0.691315
Epoch 226
Validation auc = 0.644131
Epoch 227
Validation auc = 0.640845
Epoch 228
Validation auc = 0.699296
Epoch 229
Loss = 2.4783e+00, PNorm = 83.0878, GNorm = 238.8560, lr_0 = 6.8364e-04
Validation auc = 0.699531
Epoch 230
Validation auc = 0.653521
Epoch 231
Validation auc = 0.717371
Epoch 232
Validation auc = 0.719484
Epoch 233
Validation auc = 0.718310
Epoch 234
Loss = 5.7256e-01, PNorm = 83.2632, GNorm = 186.3304, lr_0 = 6.7779e-04
Validation auc = 0.622300
Epoch 235
Validation auc = 0.655399
Epoch 236
Validation auc = 0.718545
Epoch 237
Validation auc = 0.725587
Epoch 238
Validation auc = 0.654225
Epoch 239
Loss = 2.8949e+00, PNorm = 83.4705, GNorm = 164.7272, lr_0 = 6.7200e-04
Validation auc = 0.650235
Epoch 240
Validation auc = 0.651878
Epoch 241
Validation auc = 0.692254
Epoch 242
Validation auc = 0.699531
Epoch 243
Validation auc = 0.680282
Epoch 244
Loss = 1.5371e-01, PNorm = 83.6911, GNorm = 42.0977, lr_0 = 6.6625e-04
Validation auc = 0.677230
Epoch 245
Validation auc = 0.672535
Epoch 246
Validation auc = 0.674413
Epoch 247
Validation auc = 0.663146
Epoch 248
Validation auc = 0.686150
Epoch 249
Loss = 7.4551e-01, PNorm = 83.9025, GNorm = 156.1898, lr_0 = 6.6056e-04
Validation auc = 0.680986
Epoch 250
Validation auc = 0.674883
Epoch 251
Validation auc = 0.696244
Epoch 252
Validation auc = 0.705399
Epoch 253
Validation auc = 0.709624
Epoch 254
Loss = 1.8613e+00, PNorm = 84.1035, GNorm = 390.7273, lr_0 = 6.5491e-04
Validation auc = 0.658451
Epoch 255
Validation auc = 0.646948
Epoch 256
Validation auc = 0.670188
Epoch 257
Validation auc = 0.637089
Epoch 258
Validation auc = 0.663380
Epoch 259
Loss = 1.4185e+00, PNorm = 84.2555, GNorm = 174.1264, lr_0 = 6.4931e-04
Validation auc = 0.675587
Epoch 260
Validation auc = 0.707042
Epoch 261
Validation auc = 0.702817
Epoch 262
Validation auc = 0.704930
Epoch 263
Validation auc = 0.698826
Epoch 264
Loss = 2.1734e+00, PNorm = 84.3902, GNorm = 259.3246, lr_0 = 6.4376e-04
Validation auc = 0.681925
Epoch 265
Validation auc = 0.650469
Epoch 266
Validation auc = 0.637793
Epoch 267
Validation auc = 0.698592
Epoch 268
Validation auc = 0.696244
Epoch 269
Loss = 8.4493e-01, PNorm = 84.5535, GNorm = 50.7841, lr_0 = 6.3826e-04
Validation auc = 0.642254
Epoch 270
Validation auc = 0.660563
Epoch 271
Validation auc = 0.661737
Epoch 272
Validation auc = 0.648592
Epoch 273
Validation auc = 0.677700
Epoch 274
Loss = 3.6666e-01, PNorm = 84.6982, GNorm = 81.3925, lr_0 = 6.3280e-04
Validation auc = 0.703756
Epoch 275
Validation auc = 0.674648
Epoch 276
Validation auc = 0.643192
Epoch 277
Validation auc = 0.669014
Epoch 278
Validation auc = 0.685681
Epoch 279
Loss = 1.2047e+00, PNorm = 84.9287, GNorm = 83.7413, lr_0 = 6.2739e-04
Validation auc = 0.694366
Epoch 280
Validation auc = 0.696244
Epoch 281
Validation auc = 0.676761
Epoch 282
Validation auc = 0.722535
Epoch 283
Validation auc = 0.718779
Epoch 284
Loss = 4.3287e-01, PNorm = 85.1948, GNorm = 228.9739, lr_0 = 6.2203e-04
Validation auc = 0.712207
Epoch 285
Validation auc = 0.659624
Epoch 286
Validation auc = 0.672300
Epoch 287
Validation auc = 0.666901
Epoch 288
Validation auc = 0.653052
Epoch 289
Loss = 6.1958e-01, PNorm = 85.4743, GNorm = 135.3392, lr_0 = 6.1671e-04
Validation auc = 0.653991
Epoch 290
Validation auc = 0.692019
Epoch 291
Validation auc = 0.705399
Epoch 292
Validation auc = 0.654930
Epoch 293
Validation auc = 0.652347
Epoch 294
Loss = 1.2947e+00, PNorm = 85.6870, GNorm = 162.6288, lr_0 = 6.1144e-04
Validation auc = 0.698592
Epoch 295
Validation auc = 0.696479
Epoch 296
Validation auc = 0.630986
Epoch 297
Validation auc = 0.625822
Epoch 298
Validation auc = 0.667606
Epoch 299
Loss = 4.0033e+00, PNorm = 85.9110, GNorm = 556.8784, lr_0 = 6.0621e-04
Validation auc = 0.705164
Model 0 best validation auc = 0.768075 on epoch 163
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.829922
Ensemble test auc = 0.829922
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.662207
Epoch 1
Validation auc = 0.635211
Epoch 2
Validation auc = 0.513380
Epoch 3
Validation auc = 0.667840
Epoch 4
Loss = 2.8649e+01, PNorm = 68.3637, GNorm = 2222.4227, lr_0 = 6.5000e-04
Validation auc = 0.607042
Epoch 5
Validation auc = 0.633568
Epoch 6
Validation auc = 0.704460
Epoch 7
Validation auc = 0.607981
Epoch 8
Validation auc = 0.706103
Epoch 9
Loss = 2.6902e+01, PNorm = 68.4920, GNorm = 1680.6000, lr_0 = 9.9743e-04
Validation auc = 0.707981
Epoch 10
Validation auc = 0.639437
Epoch 11
Validation auc = 0.728638
Epoch 12
Validation auc = 0.711737
Epoch 13
Validation auc = 0.725117
Epoch 14
Loss = 7.0610e+00, PNorm = 68.6358, GNorm = 462.0563, lr_0 = 9.8890e-04
Validation auc = 0.720188
Epoch 15
Validation auc = 0.747653
Epoch 16
Validation auc = 0.736150
Epoch 17
Validation auc = 0.698122
Epoch 18
Validation auc = 0.652113
Epoch 19
Loss = 1.1961e+01, PNorm = 68.7662, GNorm = 1756.1460, lr_0 = 9.8045e-04
Validation auc = 0.689437
Epoch 20
Validation auc = 0.737324
Epoch 21
Validation auc = 0.698122
Epoch 22
Validation auc = 0.684977
Epoch 23
Validation auc = 0.686385
Epoch 24
Loss = 1.1233e+01, PNorm = 68.9040, GNorm = 409.5777, lr_0 = 9.7207e-04
Validation auc = 0.702347
Epoch 25
Validation auc = 0.746244
Epoch 26
Validation auc = 0.662441
Epoch 27
Validation auc = 0.747887
Epoch 28
Validation auc = 0.749296
Epoch 29
Loss = 4.8298e+00, PNorm = 69.0856, GNorm = 680.5282, lr_0 = 9.6376e-04
Validation auc = 0.749296
Epoch 30
Validation auc = 0.704225
Epoch 31
Validation auc = 0.661502
Epoch 32
Validation auc = 0.635681
Epoch 33
Validation auc = 0.717371
Epoch 34
Loss = 6.6649e+00, PNorm = 69.2096, GNorm = 326.8015, lr_0 = 9.5552e-04
Validation auc = 0.731925
Epoch 35
Validation auc = 0.749061
Epoch 36
Validation auc = 0.720892
Epoch 37
Validation auc = 0.749061
Epoch 38
Validation auc = 0.717840
Epoch 39
Loss = 8.1165e+00, PNorm = 69.4330, GNorm = 314.3544, lr_0 = 9.4735e-04
Validation auc = 0.719953
Epoch 40
Validation auc = 0.691080
Epoch 41
Validation auc = 0.667840
Epoch 42
Validation auc = 0.642723
Epoch 43
Validation auc = 0.724413
Epoch 44
Loss = 1.0598e+01, PNorm = 69.7167, GNorm = 1700.7886, lr_0 = 9.3925e-04
Validation auc = 0.740845
Epoch 45
Validation auc = 0.732629
Epoch 46
Validation auc = 0.724648
Epoch 47
Validation auc = 0.706808
Epoch 48
Validation auc = 0.699765
Epoch 49
Loss = 1.4653e+01, PNorm = 69.9883, GNorm = 1887.0957, lr_0 = 9.3122e-04
Validation auc = 0.608920
Epoch 50
Validation auc = 0.674648
Epoch 51
Validation auc = 0.668075
Epoch 52
Validation auc = 0.693897
Epoch 53
Validation auc = 0.685681
Epoch 54
Loss = 9.9240e+00, PNorm = 70.3413, GNorm = 902.2024, lr_0 = 9.2326e-04
Validation auc = 0.679812
Epoch 55
Validation auc = 0.683333
Epoch 56
Validation auc = 0.670423
Epoch 57
Validation auc = 0.684742
Epoch 58
Validation auc = 0.642254
Epoch 59
Loss = 8.9368e+00, PNorm = 70.7254, GNorm = 223.1862, lr_0 = 9.1537e-04
Validation auc = 0.708685
Epoch 60
Validation auc = 0.660094
Epoch 61
Validation auc = 0.729577
Epoch 62
Validation auc = 0.736620
Epoch 63
Validation auc = 0.686385
Epoch 64
Loss = 1.3606e+01, PNorm = 71.0129, GNorm = 1728.4464, lr_0 = 9.0754e-04
Validation auc = 0.735446
Epoch 65
Validation auc = 0.566432
Epoch 66
Validation auc = 0.617606
Epoch 67
Validation auc = 0.713146
Epoch 68
Validation auc = 0.584977
Epoch 69
Loss = 1.2402e+01, PNorm = 71.5057, GNorm = 183.2679, lr_0 = 8.9978e-04
Validation auc = 0.664085
Epoch 70
Validation auc = 0.660094
Epoch 71
Validation auc = 0.704930
Epoch 72
Validation auc = 0.776761
Epoch 73
Validation auc = 0.768075
Epoch 74
Loss = 8.0748e+00, PNorm = 71.9549, GNorm = 440.5505, lr_0 = 8.9209e-04
Validation auc = 0.666667
Epoch 75
Validation auc = 0.693427
Epoch 76
Validation auc = 0.735211
Epoch 77
Validation auc = 0.638967
Epoch 78
Validation auc = 0.692723
Epoch 79
Loss = 9.2912e+00, PNorm = 72.1354, GNorm = 1049.4073, lr_0 = 8.8447e-04
Validation auc = 0.690376
Epoch 80
Validation auc = 0.683099
Epoch 81
Validation auc = 0.669014
Epoch 82
Validation auc = 0.738498
Epoch 83
Validation auc = 0.732864
Epoch 84
Loss = 3.4932e+00, PNorm = 72.3960, GNorm = 570.9297, lr_0 = 8.7691e-04
Validation auc = 0.680516
Epoch 85
Validation auc = 0.761502
Epoch 86
Validation auc = 0.677934
Epoch 87
Validation auc = 0.634977
Epoch 88
Validation auc = 0.705634
Epoch 89
Loss = 7.4110e+00, PNorm = 72.6182, GNorm = 834.0059, lr_0 = 8.6941e-04
Validation auc = 0.757981
Epoch 90
Validation auc = 0.669484
Epoch 91
Validation auc = 0.743662
Epoch 92
Validation auc = 0.678873
Epoch 93
Validation auc = 0.697653
Epoch 94
Loss = 2.1793e+00, PNorm = 72.7863, GNorm = 171.1770, lr_0 = 8.6198e-04
Validation auc = 0.718779
Epoch 95
Validation auc = 0.632864
Epoch 96
Validation auc = 0.672300
Epoch 97
Validation auc = 0.681925
Epoch 98
Validation auc = 0.746244
Epoch 99
Loss = 5.3568e+00, PNorm = 72.9740, GNorm = 719.4558, lr_0 = 8.5461e-04
Validation auc = 0.655164
Epoch 100
Validation auc = 0.722770
Epoch 101
Validation auc = 0.740376
Epoch 102
Validation auc = 0.746948
Epoch 103
Validation auc = 0.727230
Epoch 104
Loss = 4.3743e+00, PNorm = 73.2082, GNorm = 554.9990, lr_0 = 8.4730e-04
Validation auc = 0.734038
Epoch 105
Validation auc = 0.714554
Epoch 106
Validation auc = 0.738498
Epoch 107
Validation auc = 0.754225
Epoch 108
Validation auc = 0.726995
Epoch 109
Loss = 4.7370e+00, PNorm = 73.4075, GNorm = 1018.4827, lr_0 = 8.4006e-04
Validation auc = 0.761502
Epoch 110
Validation auc = 0.700000
Epoch 111
Validation auc = 0.760329
Epoch 112
Validation auc = 0.687559
Epoch 113
Validation auc = 0.735446
Epoch 114
Loss = 1.0861e+01, PNorm = 73.5842, GNorm = 1724.8199, lr_0 = 8.3288e-04
Validation auc = 0.744601
Epoch 115
Validation auc = 0.649765
Epoch 116
Validation auc = 0.742958
Epoch 117
Validation auc = 0.749531
Epoch 118
Validation auc = 0.651408
Epoch 119
Loss = 1.0686e+01, PNorm = 73.7834, GNorm = 578.1091, lr_0 = 8.2576e-04
Validation auc = 0.776995
Epoch 120
Validation auc = 0.776995
Epoch 121
Validation auc = 0.721596
Epoch 122
Validation auc = 0.737559
Epoch 123
Validation auc = 0.717606
Epoch 124
Loss = 6.4332e+00, PNorm = 74.0481, GNorm = 1068.8953, lr_0 = 8.1870e-04
Validation auc = 0.679108
Epoch 125
Validation auc = 0.689671
Epoch 126
Validation auc = 0.696009
Epoch 127
Validation auc = 0.680986
Epoch 128
Validation auc = 0.694601
Epoch 129
Loss = 7.8171e+00, PNorm = 74.3327, GNorm = 1359.1670, lr_0 = 8.1170e-04
Validation auc = 0.651643
Epoch 130
Validation auc = 0.714319
Epoch 131
Validation auc = 0.676291
Epoch 132
Validation auc = 0.671831
Epoch 133
Validation auc = 0.724178
Epoch 134
Loss = 1.3521e+01, PNorm = 74.7416, GNorm = 1632.0136, lr_0 = 8.0476e-04
Validation auc = 0.709390
Epoch 135
Validation auc = 0.666901
Epoch 136
Validation auc = 0.727700
Epoch 137
Validation auc = 0.722066
Epoch 138
Validation auc = 0.709390
Epoch 139
Loss = 4.4588e+00, PNorm = 75.0128, GNorm = 152.0306, lr_0 = 7.9788e-04
Validation auc = 0.729577
Epoch 140
Validation auc = 0.726995
Epoch 141
Validation auc = 0.712207
Epoch 142
Validation auc = 0.715023
Epoch 143
Validation auc = 0.713380
Epoch 144
Loss = 4.6728e+00, PNorm = 75.2388, GNorm = 596.4089, lr_0 = 7.9106e-04
Validation auc = 0.700704
Epoch 145
Validation auc = 0.694836
Epoch 146
Validation auc = 0.692019
Epoch 147
Validation auc = 0.676056
Epoch 148
Validation auc = 0.661502
Epoch 149
Loss = 5.4307e+00, PNorm = 75.4398, GNorm = 341.8956, lr_0 = 7.8430e-04
Validation auc = 0.719249
Epoch 150
Validation auc = 0.705634
Epoch 151
Validation auc = 0.705634
Epoch 152
Validation auc = 0.716197
Epoch 153
Validation auc = 0.717606
Epoch 154
Loss = 5.7360e+00, PNorm = 75.7106, GNorm = 897.6780, lr_0 = 7.7759e-04
Validation auc = 0.705399
Epoch 155
Validation auc = 0.718075
Epoch 156
Validation auc = 0.744366
Epoch 157
Validation auc = 0.653991
Epoch 158
Validation auc = 0.696714
Epoch 159
Loss = 4.8206e+00, PNorm = 75.9806, GNorm = 564.1401, lr_0 = 7.7095e-04
Validation auc = 0.622535
Epoch 160
Validation auc = 0.755634
Epoch 161
Validation auc = 0.685446
Epoch 162
Validation auc = 0.746479
Epoch 163
Validation auc = 0.668075
Epoch 164
Loss = 1.2532e+01, PNorm = 76.4413, GNorm = 328.6956, lr_0 = 7.6436e-04
Validation auc = 0.746009
Epoch 165
Validation auc = 0.665493
Epoch 166
Validation auc = 0.707277
Epoch 167
Validation auc = 0.717840
Epoch 168
Validation auc = 0.742254
Epoch 169
Loss = 6.2234e+00, PNorm = 76.8654, GNorm = 903.1595, lr_0 = 7.5782e-04
Validation auc = 0.737089
Epoch 170
Validation auc = 0.713146
Epoch 171
Validation auc = 0.771831
Epoch 172
Validation auc = 0.707512
Epoch 173
Validation auc = 0.747887
Epoch 174
Loss = 5.9937e+00, PNorm = 77.1120, GNorm = 524.4976, lr_0 = 7.5134e-04
Validation auc = 0.745775
Epoch 175
Validation auc = 0.765728
Epoch 176
Validation auc = 0.618310
Epoch 177
Validation auc = 0.733568
Epoch 178
Validation auc = 0.670657
Epoch 179
Loss = 4.8854e+00, PNorm = 77.3219, GNorm = 451.5765, lr_0 = 7.4492e-04
Validation auc = 0.687324
Epoch 180
Validation auc = 0.704225
Epoch 181
Validation auc = 0.721127
Epoch 182
Validation auc = 0.703052
Epoch 183
Validation auc = 0.713615
Epoch 184
Loss = 5.6332e+00, PNorm = 77.5903, GNorm = 180.5542, lr_0 = 7.3855e-04
Validation auc = 0.719953
Epoch 185
Validation auc = 0.721596
Epoch 186
Validation auc = 0.705634
Epoch 187
Validation auc = 0.705634
Epoch 188
Validation auc = 0.716197
Epoch 189
Loss = 7.0055e+00, PNorm = 77.7176, GNorm = 337.5399, lr_0 = 7.3224e-04
Validation auc = 0.710329
Epoch 190
Validation auc = 0.686854
Epoch 191
Validation auc = 0.690141
Epoch 192
Validation auc = 0.713380
Epoch 193
Validation auc = 0.655164
Epoch 194
Loss = 4.4229e+00, PNorm = 77.8370, GNorm = 152.2468, lr_0 = 7.2598e-04
Validation auc = 0.722066
Epoch 195
Validation auc = 0.711972
Epoch 196
Validation auc = 0.680516
Epoch 197
Validation auc = 0.704225
Epoch 198
Validation auc = 0.741315
Epoch 199
Loss = 5.9248e+00, PNorm = 78.0215, GNorm = 175.1508, lr_0 = 7.1977e-04
Validation auc = 0.708920
Epoch 200
Validation auc = 0.703991
Epoch 201
Validation auc = 0.749531
Epoch 202
Validation auc = 0.758451
Epoch 203
Validation auc = 0.746244
Epoch 204
Loss = 3.0372e+00, PNorm = 78.2221, GNorm = 145.6185, lr_0 = 7.1362e-04
Validation auc = 0.737793
Epoch 205
Validation auc = 0.743427
Epoch 206
Validation auc = 0.715728
Epoch 207
Validation auc = 0.745775
Epoch 208
Validation auc = 0.731690
Epoch 209
Loss = 2.6501e+00, PNorm = 78.4708, GNorm = 401.5392, lr_0 = 7.0752e-04
Validation auc = 0.708216
Epoch 210
Validation auc = 0.697183
Epoch 211
Validation auc = 0.735681
Epoch 212
Validation auc = 0.732629
Epoch 213
Validation auc = 0.713146
Epoch 214
Loss = 1.6374e+00, PNorm = 78.6497, GNorm = 74.3271, lr_0 = 7.0147e-04
Validation auc = 0.694601
Epoch 215
Validation auc = 0.721831
Epoch 216
Validation auc = 0.716901
Epoch 217
Validation auc = 0.684977
Epoch 218
Validation auc = 0.697183
Epoch 219
Loss = 2.2091e+00, PNorm = 78.9110, GNorm = 184.1712, lr_0 = 6.9548e-04
Validation auc = 0.712207
Epoch 220
Validation auc = 0.669718
Epoch 221
Validation auc = 0.674178
Epoch 222
Validation auc = 0.670423
Epoch 223
Validation auc = 0.673709
Epoch 224
Loss = 4.3459e+00, PNorm = 79.1868, GNorm = 300.0848, lr_0 = 6.8953e-04
Validation auc = 0.648592
Epoch 225
Validation auc = 0.667136
Epoch 226
Validation auc = 0.668545
Epoch 227
Validation auc = 0.677465
Epoch 228
Validation auc = 0.683803
Epoch 229
Loss = 1.8226e+00, PNorm = 79.5216, GNorm = 203.8537, lr_0 = 6.8364e-04
Validation auc = 0.658216
Epoch 230
Validation auc = 0.667136
Epoch 231
Validation auc = 0.650235
Epoch 232
Validation auc = 0.673005
Epoch 233
Validation auc = 0.677230
Epoch 234
Loss = 1.4701e+00, PNorm = 79.8390, GNorm = 449.9838, lr_0 = 6.7779e-04
Validation auc = 0.678873
Epoch 235
Validation auc = 0.698592
Epoch 236
Validation auc = 0.704225
Epoch 237
Validation auc = 0.708685
Epoch 238
Validation auc = 0.720188
Epoch 239
Loss = 2.2132e+00, PNorm = 80.1766, GNorm = 230.4419, lr_0 = 6.7200e-04
Validation auc = 0.695775
Epoch 240
Validation auc = 0.710094
Epoch 241
Validation auc = 0.737089
Epoch 242
Validation auc = 0.726995
Epoch 243
Validation auc = 0.705869
Epoch 244
Loss = 1.4822e+00, PNorm = 80.4574, GNorm = 468.8149, lr_0 = 6.6625e-04
Validation auc = 0.728404
Epoch 245
Validation auc = 0.723005
Epoch 246
Validation auc = 0.712441
Epoch 247
Validation auc = 0.707512
Epoch 248
Validation auc = 0.699296
Epoch 249
Loss = 1.3634e+00, PNorm = 80.6660, GNorm = 155.1900, lr_0 = 6.6056e-04
Validation auc = 0.740845
Epoch 250
Validation auc = 0.729577
Epoch 251
Validation auc = 0.716901
Epoch 252
Validation auc = 0.692958
Epoch 253
Validation auc = 0.689437
Epoch 254
Loss = 1.1142e+00, PNorm = 80.8428, GNorm = 130.3675, lr_0 = 6.5491e-04
Validation auc = 0.719014
Epoch 255
Validation auc = 0.712911
Epoch 256
Validation auc = 0.699296
Epoch 257
Validation auc = 0.673239
Epoch 258
Validation auc = 0.673709
Epoch 259
Loss = 1.9125e+00, PNorm = 81.0174, GNorm = 253.4904, lr_0 = 6.4931e-04
Validation auc = 0.665493
Epoch 260
Validation auc = 0.674178
Epoch 261
Validation auc = 0.695305
Epoch 262
Validation auc = 0.686150
Epoch 263
Validation auc = 0.681221
Epoch 264
Loss = 9.9696e-01, PNorm = 81.1739, GNorm = 195.7074, lr_0 = 6.4376e-04
Validation auc = 0.704930
Epoch 265
Validation auc = 0.727700
Epoch 266
Validation auc = 0.703521
Epoch 267
Validation auc = 0.673474
Epoch 268
Validation auc = 0.702113
Epoch 269
Loss = 3.2700e-01, PNorm = 81.3435, GNorm = 182.9388, lr_0 = 6.3826e-04
Validation auc = 0.730047
Epoch 270
Validation auc = 0.744131
Epoch 271
Validation auc = 0.677700
Epoch 272
Validation auc = 0.716901
Epoch 273
Validation auc = 0.726056
Epoch 274
Loss = 1.3959e+00, PNorm = 81.5238, GNorm = 79.8995, lr_0 = 6.3280e-04
Validation auc = 0.695305
Epoch 275
Validation auc = 0.705164
Epoch 276
Validation auc = 0.729108
Epoch 277
Validation auc = 0.700704
Epoch 278
Validation auc = 0.670892
Epoch 279
Loss = 2.5374e+00, PNorm = 81.7691, GNorm = 150.9771, lr_0 = 6.2739e-04
Validation auc = 0.695305
Epoch 280
Validation auc = 0.714554
Epoch 281
Validation auc = 0.672770
Epoch 282
Validation auc = 0.670423
Epoch 283
Validation auc = 0.688263
Epoch 284
Loss = 2.2564e+00, PNorm = 82.0437, GNorm = 287.1075, lr_0 = 6.2203e-04
Validation auc = 0.662441
Epoch 285
Validation auc = 0.671362
Epoch 286
Validation auc = 0.682160
Epoch 287
Validation auc = 0.665962
Epoch 288
Validation auc = 0.663615
Epoch 289
Loss = 8.9563e-01, PNorm = 82.3325, GNorm = 439.7990, lr_0 = 6.1671e-04
Validation auc = 0.675352
Epoch 290
Validation auc = 0.707746
Epoch 291
Validation auc = 0.652582
Epoch 292
Validation auc = 0.692958
Epoch 293
Validation auc = 0.681925
Epoch 294
Loss = 1.1223e+00, PNorm = 82.5794, GNorm = 115.0436, lr_0 = 6.1144e-04
Validation auc = 0.652582
Epoch 295
Validation auc = 0.657746
Epoch 296
Validation auc = 0.687089
Epoch 297
Validation auc = 0.730282
Epoch 298
Validation auc = 0.714554
Epoch 299
Loss = 9.9521e-02, PNorm = 82.7099, GNorm = 213.4942, lr_0 = 6.0621e-04
Validation auc = 0.687793
Model 0 best validation auc = 0.776995 on epoch 119
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.719446
Ensemble test auc = 0.719446
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.600000
Epoch 1
Validation auc = 0.475587
Epoch 2
Validation auc = 0.645540
Epoch 3
Validation auc = 0.656103
Epoch 4
Loss = 2.3413e+01, PNorm = 68.3645, GNorm = 2045.4525, lr_0 = 6.5000e-04
Validation auc = 0.712676
Epoch 5
Validation auc = 0.757042
Epoch 6
Validation auc = 0.661502
Epoch 7
Validation auc = 0.721127
Epoch 8
Validation auc = 0.615728
Epoch 9
Loss = 2.4147e+01, PNorm = 68.5084, GNorm = 1061.5458, lr_0 = 9.9743e-04
Validation auc = 0.675587
Epoch 10
Validation auc = 0.558920
Epoch 11
Validation auc = 0.680282
Epoch 12
Validation auc = 0.615728
Epoch 13
Validation auc = 0.719014
Epoch 14
Loss = 7.6039e+00, PNorm = 68.6570, GNorm = 971.5785, lr_0 = 9.8890e-04
Validation auc = 0.712207
Epoch 15
Validation auc = 0.700000
Epoch 16
Validation auc = 0.676995
Epoch 17
Validation auc = 0.645775
Epoch 18
Validation auc = 0.623474
Epoch 19
Loss = 1.0492e+01, PNorm = 68.7734, GNorm = 1120.7784, lr_0 = 9.8045e-04
Validation auc = 0.607512
Epoch 20
Validation auc = 0.610798
Epoch 21
Validation auc = 0.665728
Epoch 22
Validation auc = 0.622535
Epoch 23
Validation auc = 0.622300
Epoch 24
Loss = 9.2500e+00, PNorm = 68.8663, GNorm = 285.8341, lr_0 = 9.7207e-04
Validation auc = 0.623239
Epoch 25
Validation auc = 0.650704
Epoch 26
Validation auc = 0.676995
Epoch 27
Validation auc = 0.668310
Epoch 28
Validation auc = 0.633333
Epoch 29
Loss = 1.7463e+01, PNorm = 69.0434, GNorm = 1651.8413, lr_0 = 9.6376e-04
Validation auc = 0.636385
Epoch 30
Validation auc = 0.655869
Epoch 31
Validation auc = 0.692488
Epoch 32
Validation auc = 0.561268
Epoch 33
Validation auc = 0.645070
Epoch 34
Loss = 2.1999e+01, PNorm = 69.1876, GNorm = 1840.5786, lr_0 = 9.5552e-04
Validation auc = 0.700704
Epoch 35
Validation auc = 0.595305
Epoch 36
Validation auc = 0.654460
Epoch 37
Validation auc = 0.651174
Epoch 38
Validation auc = 0.718310
Epoch 39
Loss = 8.9299e+00, PNorm = 69.3629, GNorm = 326.0689, lr_0 = 9.4735e-04
Validation auc = 0.679343
Epoch 40
Validation auc = 0.691315
Epoch 41
Validation auc = 0.687793
Epoch 42
Validation auc = 0.665258
Epoch 43
Validation auc = 0.667136
Epoch 44
Loss = 1.2371e+01, PNorm = 69.5490, GNorm = 585.3242, lr_0 = 9.3925e-04
Validation auc = 0.675352
Epoch 45
Validation auc = 0.665258
Epoch 46
Validation auc = 0.689202
Epoch 47
Validation auc = 0.659155
Epoch 48
Validation auc = 0.676056
Epoch 49
Loss = 1.2091e+01, PNorm = 69.8220, GNorm = 196.1528, lr_0 = 9.3122e-04
Validation auc = 0.616197
Epoch 50
Validation auc = 0.702113
Epoch 51
Validation auc = 0.712911
Epoch 52
Validation auc = 0.617136
Epoch 53
Validation auc = 0.695775
Epoch 54
Loss = 7.7959e+00, PNorm = 70.2123, GNorm = 601.8572, lr_0 = 9.2326e-04
Validation auc = 0.667606
Epoch 55
Validation auc = 0.707042
Epoch 56
Validation auc = 0.656808
Epoch 57
Validation auc = 0.608216
Epoch 58
Validation auc = 0.680282
Epoch 59
Loss = 1.8606e+01, PNorm = 70.7177, GNorm = 1342.5645, lr_0 = 9.1537e-04
Validation auc = 0.728404
Epoch 60
Validation auc = 0.664319
Epoch 61
Validation auc = 0.689906
Epoch 62
Validation auc = 0.723944
Epoch 63
Validation auc = 0.661972
Epoch 64
Loss = 1.6127e+01, PNorm = 71.0933, GNorm = 389.5743, lr_0 = 9.0754e-04
Validation auc = 0.686150
Epoch 65
Validation auc = 0.693427
Epoch 66
Validation auc = 0.702113
Epoch 67
Validation auc = 0.708216
Epoch 68
Validation auc = 0.668075
Epoch 69
Loss = 8.1011e+00, PNorm = 71.3801, GNorm = 203.2882, lr_0 = 8.9978e-04
Validation auc = 0.691080
Epoch 70
Validation auc = 0.676526
Epoch 71
Validation auc = 0.688263
Epoch 72
Validation auc = 0.643897
Epoch 73
Validation auc = 0.680516
Epoch 74
Loss = 3.5612e+00, PNorm = 71.5691, GNorm = 412.4853, lr_0 = 8.9209e-04
Validation auc = 0.726291
Epoch 75
Validation auc = 0.701643
Epoch 76
Validation auc = 0.710329
Epoch 77
Validation auc = 0.730516
Epoch 78
Validation auc = 0.738028
Epoch 79
Loss = 1.0306e+01, PNorm = 71.7511, GNorm = 345.2022, lr_0 = 8.8447e-04
Validation auc = 0.684272
Epoch 80
Validation auc = 0.720657
Epoch 81
Validation auc = 0.724413
Epoch 82
Validation auc = 0.711502
Epoch 83
Validation auc = 0.775117
Epoch 84
Loss = 5.8438e+00, PNorm = 71.8674, GNorm = 510.4134, lr_0 = 8.7691e-04
Validation auc = 0.719484
Epoch 85
Validation auc = 0.748592
Epoch 86
Validation auc = 0.680751
Epoch 87
Validation auc = 0.690376
Epoch 88
Validation auc = 0.659859
Epoch 89
Loss = 7.8127e+00, PNorm = 72.0182, GNorm = 1737.5920, lr_0 = 8.6941e-04
Validation auc = 0.692958
Epoch 90
Validation auc = 0.714789
Epoch 91
Validation auc = 0.696009
Epoch 92
Validation auc = 0.688967
Epoch 93
Validation auc = 0.735446
Epoch 94
Loss = 5.7044e+00, PNorm = 72.2186, GNorm = 596.6441, lr_0 = 8.6198e-04
Validation auc = 0.660329
Epoch 95
Validation auc = 0.728873
Epoch 96
Validation auc = 0.716432
Epoch 97
Validation auc = 0.675587
Epoch 98
Validation auc = 0.730282
Epoch 99
Loss = 1.1092e+01, PNorm = 72.4233, GNorm = 1750.4676, lr_0 = 8.5461e-04
Validation auc = 0.738263
Epoch 100
Validation auc = 0.678169
Epoch 101
Validation auc = 0.732394
Epoch 102
Validation auc = 0.714789
Epoch 103
Validation auc = 0.744131
Epoch 104
Loss = 8.2378e+00, PNorm = 72.5445, GNorm = 768.6494, lr_0 = 8.4730e-04
Validation auc = 0.752113
Epoch 105
Validation auc = 0.718075
Epoch 106
Validation auc = 0.752817
Epoch 107
Validation auc = 0.700939
Epoch 108
Validation auc = 0.756338
Epoch 109
Loss = 1.0026e+01, PNorm = 72.7199, GNorm = 551.3067, lr_0 = 8.4006e-04
Validation auc = 0.678404
Epoch 110
Validation auc = 0.715023
Epoch 111
Validation auc = 0.682864
Epoch 112
Validation auc = 0.720657
Epoch 113
Validation auc = 0.722300
Epoch 114
Loss = 4.6551e+00, PNorm = 72.8921, GNorm = 205.2359, lr_0 = 8.3288e-04
Validation auc = 0.684507
Epoch 115
Validation auc = 0.690141
Epoch 116
Validation auc = 0.686150
Epoch 117
Validation auc = 0.751878
Epoch 118
Validation auc = 0.699061
Epoch 119
Loss = 2.6325e+00, PNorm = 73.1236, GNorm = 577.6325, lr_0 = 8.2576e-04
Validation auc = 0.730986
Epoch 120
Validation auc = 0.733803
Epoch 121
Validation auc = 0.730047
Epoch 122
Validation auc = 0.734272
Epoch 123
Validation auc = 0.728169
Epoch 124
Loss = 3.3514e+00, PNorm = 73.4050, GNorm = 300.3415, lr_0 = 8.1870e-04
Validation auc = 0.717606
Epoch 125
Validation auc = 0.715962
Epoch 126
Validation auc = 0.705634
Epoch 127
Validation auc = 0.706573
Epoch 128
Validation auc = 0.702817
Epoch 129
Loss = 2.7341e+00, PNorm = 73.5968, GNorm = 91.4264, lr_0 = 8.1170e-04
Validation auc = 0.694131
Epoch 130
Validation auc = 0.696714
Epoch 131
Validation auc = 0.661502
Epoch 132
Validation auc = 0.710798
Epoch 133
Validation auc = 0.705634
Epoch 134
Loss = 2.1284e+00, PNorm = 73.8083, GNorm = 576.5848, lr_0 = 8.0476e-04
Validation auc = 0.718310
Epoch 135
Validation auc = 0.712207
Epoch 136
Validation auc = 0.713146
Epoch 137
Validation auc = 0.738967
Epoch 138
Validation auc = 0.690845
Epoch 139
Loss = 8.1260e+00, PNorm = 74.0499, GNorm = 848.8499, lr_0 = 7.9788e-04
Validation auc = 0.720892
Epoch 140
Validation auc = 0.741080
Epoch 141
Validation auc = 0.686620
Epoch 142
Validation auc = 0.700939
Epoch 143
Validation auc = 0.676056
Epoch 144
Loss = 4.7150e+00, PNorm = 74.1890, GNorm = 201.8191, lr_0 = 7.9106e-04
Validation auc = 0.757512
Epoch 145
Validation auc = 0.704460
Epoch 146
Validation auc = 0.755399
Epoch 147
Validation auc = 0.693192
Epoch 148
Validation auc = 0.763380
Epoch 149
Loss = 2.0178e+00, PNorm = 74.3810, GNorm = 170.4198, lr_0 = 7.8430e-04
Validation auc = 0.721831
Epoch 150
Validation auc = 0.757746
Epoch 151
Validation auc = 0.742723
Epoch 152
Validation auc = 0.711502
Epoch 153
Validation auc = 0.757981
Epoch 154
Loss = 6.0256e+00, PNorm = 74.5525, GNorm = 137.5832, lr_0 = 7.7759e-04
Validation auc = 0.726056
Epoch 155
Validation auc = 0.685211
Epoch 156
Validation auc = 0.714789
Epoch 157
Validation auc = 0.676291
Epoch 158
Validation auc = 0.769484
Epoch 159
Loss = 9.3037e+00, PNorm = 74.8065, GNorm = 1482.7884, lr_0 = 7.7095e-04
Validation auc = 0.739202
Epoch 160
Validation auc = 0.650704
Epoch 161
Validation auc = 0.741549
Epoch 162
Validation auc = 0.615728
Epoch 163
Validation auc = 0.661972
Epoch 164
Loss = 2.2527e+01, PNorm = 75.3135, GNorm = 1699.9890, lr_0 = 7.6436e-04
Validation auc = 0.733568
Epoch 165
Validation auc = 0.666901
Epoch 166
Validation auc = 0.754930
Epoch 167
Validation auc = 0.675117
Epoch 168
Validation auc = 0.742723
Epoch 169
Loss = 5.8178e+00, PNorm = 75.9121, GNorm = 738.6581, lr_0 = 7.5782e-04
Validation auc = 0.671831
Epoch 170
Validation auc = 0.724413
Epoch 171
Validation auc = 0.700000
Epoch 172
Validation auc = 0.736385
Epoch 173
Validation auc = 0.697418
Epoch 174
Loss = 6.5180e+00, PNorm = 76.3812, GNorm = 169.5200, lr_0 = 7.5134e-04
Validation auc = 0.694601
Epoch 175
Validation auc = 0.750000
Epoch 176
Validation auc = 0.715962
Epoch 177
Validation auc = 0.648122
Epoch 178
Validation auc = 0.746948
Epoch 179
Loss = 7.0649e+00, PNorm = 76.7665, GNorm = 815.4623, lr_0 = 7.4492e-04
Validation auc = 0.718075
Epoch 180
Validation auc = 0.690141
Epoch 181
Validation auc = 0.749296
Epoch 182
Validation auc = 0.704460
Epoch 183
Validation auc = 0.679343
Epoch 184
Loss = 2.7152e+00, PNorm = 77.1318, GNorm = 341.2099, lr_0 = 7.3855e-04
Validation auc = 0.661972
Epoch 185
Validation auc = 0.679577
Epoch 186
Validation auc = 0.699296
Epoch 187
Validation auc = 0.692958
Epoch 188
Validation auc = 0.709624
Epoch 189
Loss = 3.7028e+00, PNorm = 77.4140, GNorm = 300.2565, lr_0 = 7.3224e-04
Validation auc = 0.687089
Epoch 190
Validation auc = 0.730986
Epoch 191
Validation auc = 0.696479
Epoch 192
Validation auc = 0.663146
Epoch 193
Validation auc = 0.732394
Epoch 194
Loss = 2.0033e+00, PNorm = 77.7187, GNorm = 211.8011, lr_0 = 7.2598e-04
Validation auc = 0.689906
Epoch 195
Validation auc = 0.701878
Epoch 196
Validation auc = 0.761502
Epoch 197
Validation auc = 0.764319
Epoch 198
Validation auc = 0.663850
Epoch 199
Loss = 4.7713e+00, PNorm = 77.9926, GNorm = 575.2049, lr_0 = 7.1977e-04
Validation auc = 0.753756
Epoch 200
Validation auc = 0.713146
Epoch 201
Validation auc = 0.689906
Epoch 202
Validation auc = 0.697183
Epoch 203
Validation auc = 0.754930
Epoch 204
Loss = 2.4180e+00, PNorm = 78.3497, GNorm = 335.5227, lr_0 = 7.1362e-04
Validation auc = 0.758920
Epoch 205
Validation auc = 0.726056
Epoch 206
Validation auc = 0.723239
Epoch 207
Validation auc = 0.735446
Epoch 208
Validation auc = 0.754930
Epoch 209
Loss = 1.8042e+00, PNorm = 78.6688, GNorm = 463.4245, lr_0 = 7.0752e-04
Validation auc = 0.742723
Epoch 210
Validation auc = 0.698592
Epoch 211
Validation auc = 0.763615
Epoch 212
Validation auc = 0.744131
Epoch 213
Validation auc = 0.703052
Epoch 214
Loss = 4.9511e+00, PNorm = 78.9412, GNorm = 910.8986, lr_0 = 7.0147e-04
Validation auc = 0.727230
Epoch 215
Validation auc = 0.745540
Epoch 216
Validation auc = 0.719014
Epoch 217
Validation auc = 0.707981
Epoch 218
Validation auc = 0.717840
Epoch 219
Loss = 5.1751e-01, PNorm = 79.1964, GNorm = 122.5047, lr_0 = 6.9548e-04
Validation auc = 0.724413
Epoch 220
Validation auc = 0.756808
Epoch 221
Validation auc = 0.751174
Epoch 222
Validation auc = 0.746948
Epoch 223
Validation auc = 0.705634
Epoch 224
Loss = 2.2833e+00, PNorm = 79.4254, GNorm = 348.7590, lr_0 = 6.8953e-04
Validation auc = 0.722300
Epoch 225
Validation auc = 0.653991
Epoch 226
Validation auc = 0.705399
Epoch 227
Validation auc = 0.652582
Epoch 228
Validation auc = 0.703991
Epoch 229
Loss = 3.1106e-01, PNorm = 79.6498, GNorm = 391.8935, lr_0 = 6.8364e-04
Validation auc = 0.695540
Epoch 230
Validation auc = 0.701878
Epoch 231
Validation auc = 0.739437
Epoch 232
Validation auc = 0.743192
Epoch 233
Validation auc = 0.705164
Epoch 234
Loss = 1.5723e+00, PNorm = 79.8860, GNorm = 214.5336, lr_0 = 6.7779e-04
Validation auc = 0.715728
Epoch 235
Validation auc = 0.689437
Epoch 236
Validation auc = 0.682394
Epoch 237
Validation auc = 0.691784
Epoch 238
Validation auc = 0.713615
Epoch 239
Loss = 3.3227e+00, PNorm = 80.1307, GNorm = 351.4933, lr_0 = 6.7200e-04
Validation auc = 0.689437
Epoch 240
Validation auc = 0.673474
Epoch 241
Validation auc = 0.700469
Epoch 242
Validation auc = 0.711268
Epoch 243
Validation auc = 0.684038
Epoch 244
Loss = 1.4502e+00, PNorm = 80.4178, GNorm = 302.3438, lr_0 = 6.6625e-04
Validation auc = 0.679812
Epoch 245
Validation auc = 0.711268
Epoch 246
Validation auc = 0.691784
Epoch 247
Validation auc = 0.663850
Epoch 248
Validation auc = 0.722535
Epoch 249
Loss = 2.7000e+00, PNorm = 80.7207, GNorm = 520.7025, lr_0 = 6.6056e-04
Validation auc = 0.713615
Epoch 250
Validation auc = 0.676995
Epoch 251
Validation auc = 0.721596
Epoch 252
Validation auc = 0.700235
Epoch 253
Validation auc = 0.697183
Epoch 254
Loss = 2.7928e+00, PNorm = 81.0348, GNorm = 571.7703, lr_0 = 6.5491e-04
Validation auc = 0.720657
Epoch 255
Validation auc = 0.706573
Epoch 256
Validation auc = 0.703756
Epoch 257
Validation auc = 0.719014
Epoch 258
Validation auc = 0.717606
Epoch 259
Loss = 1.3062e+00, PNorm = 81.3756, GNorm = 403.7526, lr_0 = 6.4931e-04
Validation auc = 0.692019
Epoch 260
Validation auc = 0.693427
Epoch 261
Validation auc = 0.714085
Epoch 262
Validation auc = 0.701174
Epoch 263
Validation auc = 0.693427
Epoch 264
Loss = 3.8325e-01, PNorm = 81.6770, GNorm = 466.4489, lr_0 = 6.4376e-04
Validation auc = 0.706808
Epoch 265
Validation auc = 0.726056
Epoch 266
Validation auc = 0.691080
Epoch 267
Validation auc = 0.705399
Epoch 268
Validation auc = 0.695775
Epoch 269
Loss = 5.0016e-01, PNorm = 81.9808, GNorm = 123.3423, lr_0 = 6.3826e-04
Validation auc = 0.662911
Epoch 270
Validation auc = 0.683568
Epoch 271
Validation auc = 0.688263
Epoch 272
Validation auc = 0.674178
Epoch 273
Validation auc = 0.634977
Epoch 274
Loss = 8.8900e-01, PNorm = 82.1702, GNorm = 284.1953, lr_0 = 6.3280e-04
Validation auc = 0.623944
Epoch 275
Validation auc = 0.618075
Epoch 276
Validation auc = 0.623709
Epoch 277
Validation auc = 0.648357
Epoch 278
Validation auc = 0.637089
Epoch 279
Loss = 1.0230e+00, PNorm = 82.3270, GNorm = 313.1517, lr_0 = 6.2739e-04
Validation auc = 0.635915
Epoch 280
Validation auc = 0.689906
Epoch 281
Validation auc = 0.643662
Epoch 282
Validation auc = 0.668545
Epoch 283
Validation auc = 0.670188
Epoch 284
Loss = 1.4613e+00, PNorm = 82.4850, GNorm = 134.4315, lr_0 = 6.2203e-04
Validation auc = 0.645540
Epoch 285
Validation auc = 0.673709
Epoch 286
Validation auc = 0.668779
Epoch 287
Validation auc = 0.669484
Epoch 288
Validation auc = 0.662911
Epoch 289
Loss = 1.3968e+00, PNorm = 82.7033, GNorm = 383.3735, lr_0 = 6.1671e-04
Validation auc = 0.655399
Epoch 290
Validation auc = 0.688498
Epoch 291
Validation auc = 0.665728
Epoch 292
Validation auc = 0.651878
Epoch 293
Validation auc = 0.688263
Epoch 294
Loss = 2.4449e+00, PNorm = 82.9902, GNorm = 157.1244, lr_0 = 6.1144e-04
Validation auc = 0.640610
Epoch 295
Validation auc = 0.680986
Epoch 296
Validation auc = 0.686854
Epoch 297
Validation auc = 0.622066
Epoch 298
Validation auc = 0.651643
Epoch 299
Loss = 2.3095e+00, PNorm = 83.2788, GNorm = 266.2051, lr_0 = 6.0621e-04
Validation auc = 0.697183
Model 0 best validation auc = 0.775117 on epoch 83
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.815172
Ensemble test auc = 0.815172
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.613615
Epoch 1
Validation auc = 0.487793
Epoch 2
Validation auc = 0.743192
Epoch 3
Validation auc = 0.743192
Epoch 4
Loss = 2.7186e+01, PNorm = 68.3695, GNorm = 1448.5989, lr_0 = 6.5000e-04
Validation auc = 0.533568
Epoch 5
Validation auc = 0.676995
Epoch 6
Validation auc = 0.706103
Epoch 7
Validation auc = 0.666197
Epoch 8
Validation auc = 0.611268
Epoch 9
Loss = 3.0700e+01, PNorm = 68.5379, GNorm = 410.7667, lr_0 = 9.9743e-04
Validation auc = 0.743427
Epoch 10
Validation auc = 0.720188
Epoch 11
Validation auc = 0.550704
Epoch 12
Validation auc = 0.752582
Epoch 13
Validation auc = 0.671362
Epoch 14
Loss = 2.9077e+01, PNorm = 68.7146, GNorm = 516.2112, lr_0 = 9.8890e-04
Validation auc = 0.568779
Epoch 15
Validation auc = 0.631221
Epoch 16
Validation auc = 0.688263
Epoch 17
Validation auc = 0.617371
Epoch 18
Validation auc = 0.668545
Epoch 19
Loss = 1.4295e+01, PNorm = 68.8492, GNorm = 466.7568, lr_0 = 9.8045e-04
Validation auc = 0.649765
Epoch 20
Validation auc = 0.701878
Epoch 21
Validation auc = 0.613615
Epoch 22
Validation auc = 0.683099
Epoch 23
Validation auc = 0.708451
Epoch 24
Loss = 9.4628e+00, PNorm = 68.9603, GNorm = 1158.3835, lr_0 = 9.7207e-04
Validation auc = 0.725822
Epoch 25
Validation auc = 0.762911
Epoch 26
Validation auc = 0.756103
Epoch 27
Validation auc = 0.729108
Epoch 28
Validation auc = 0.489202
Epoch 29
Loss = 2.6948e+01, PNorm = 69.0612, GNorm = 1915.2895, lr_0 = 9.6376e-04
Validation auc = 0.692723
Epoch 30
Validation auc = 0.723005
Epoch 31
Validation auc = 0.686854
Epoch 32
Validation auc = 0.672770
Epoch 33
Validation auc = 0.629577
Epoch 34
Loss = 1.3679e+01, PNorm = 69.1605, GNorm = 348.4174, lr_0 = 9.5552e-04
Validation auc = 0.709155
Epoch 35
Validation auc = 0.718310
Epoch 36
Validation auc = 0.680986
Epoch 37
Validation auc = 0.706103
Epoch 38
Validation auc = 0.666667
Epoch 39
Loss = 8.3307e+00, PNorm = 69.3311, GNorm = 267.9729, lr_0 = 9.4735e-04
Validation auc = 0.704460
Epoch 40
Validation auc = 0.693427
Epoch 41
Validation auc = 0.616432
Epoch 42
Validation auc = 0.662207
Epoch 43
Validation auc = 0.561972
Epoch 44
Loss = 1.7360e+01, PNorm = 69.4779, GNorm = 322.6521, lr_0 = 9.3925e-04
Validation auc = 0.525352
Epoch 45
Validation auc = 0.692254
Epoch 46
Validation auc = 0.715023
Epoch 47
Validation auc = 0.655634
Epoch 48
Validation auc = 0.576995
Epoch 49
Loss = 2.2027e+01, PNorm = 69.7529, GNorm = 1091.7815, lr_0 = 9.3122e-04
Validation auc = 0.681221
Epoch 50
Validation auc = 0.672066
Epoch 51
Validation auc = 0.733099
Epoch 52
Validation auc = 0.607042
Epoch 53
Validation auc = 0.710329
Epoch 54
Loss = 1.0794e+01, PNorm = 70.0660, GNorm = 1501.5377, lr_0 = 9.2326e-04
Validation auc = 0.715023
Epoch 55
Validation auc = 0.626056
Epoch 56
Validation auc = 0.673239
Epoch 57
Validation auc = 0.676761
Epoch 58
Validation auc = 0.668075
Epoch 59
Loss = 6.2680e+00, PNorm = 70.4518, GNorm = 545.3033, lr_0 = 9.1537e-04
Validation auc = 0.688967
Epoch 60
Validation auc = 0.675352
Epoch 61
Validation auc = 0.648357
Epoch 62
Validation auc = 0.719953
Epoch 63
Validation auc = 0.684977
Epoch 64
Loss = 7.4801e+00, PNorm = 70.8249, GNorm = 185.6471, lr_0 = 9.0754e-04
Validation auc = 0.652113
Epoch 65
Validation auc = 0.690141
Epoch 66
Validation auc = 0.701878
Epoch 67
Validation auc = 0.656103
Epoch 68
Validation auc = 0.688028
Epoch 69
Loss = 1.2469e+01, PNorm = 71.1242, GNorm = 913.7873, lr_0 = 8.9978e-04
Validation auc = 0.709624
Epoch 70
Validation auc = 0.649296
Epoch 71
Validation auc = 0.751174
Epoch 72
Validation auc = 0.732629
Epoch 73
Validation auc = 0.700000
Epoch 74
Loss = 4.0240e+00, PNorm = 71.4517, GNorm = 152.5637, lr_0 = 8.9209e-04
Validation auc = 0.748357
Epoch 75
Validation auc = 0.654930
Epoch 76
Validation auc = 0.759155
Epoch 77
Validation auc = 0.637089
Epoch 78
Validation auc = 0.594601
Epoch 79
Loss = 2.3788e+01, PNorm = 71.7594, GNorm = 1844.3191, lr_0 = 8.8447e-04
Validation auc = 0.721127
Epoch 80
Validation auc = 0.585446
Epoch 81
Validation auc = 0.704695
Epoch 82
Validation auc = 0.684272
Epoch 83
Validation auc = 0.695775
Epoch 84
Loss = 1.1307e+01, PNorm = 72.1425, GNorm = 1170.2007, lr_0 = 8.7691e-04
Validation auc = 0.704225
Epoch 85
Validation auc = 0.736150
Epoch 86
Validation auc = 0.734272
Epoch 87
Validation auc = 0.711033
Epoch 88
Validation auc = 0.745540
Epoch 89
Loss = 1.2831e+01, PNorm = 72.4952, GNorm = 571.5758, lr_0 = 8.6941e-04
Validation auc = 0.704930
Epoch 90
Validation auc = 0.719014
Epoch 91
Validation auc = 0.722770
Epoch 92
Validation auc = 0.659390
Epoch 93
Validation auc = 0.709624
Epoch 94
Loss = 8.1850e+00, PNorm = 72.7402, GNorm = 1267.2281, lr_0 = 8.6198e-04
Validation auc = 0.659155
Epoch 95
Validation auc = 0.713146
Epoch 96
Validation auc = 0.690845
Epoch 97
Validation auc = 0.741549
Epoch 98
Validation auc = 0.746948
Epoch 99
Loss = 3.7715e+00, PNorm = 72.8960, GNorm = 313.1899, lr_0 = 8.5461e-04
Validation auc = 0.736620
Epoch 100
Validation auc = 0.763146
Epoch 101
Validation auc = 0.703286
Epoch 102
Validation auc = 0.671127
Epoch 103
Validation auc = 0.740610
Epoch 104
Loss = 7.7254e+00, PNorm = 73.0926, GNorm = 339.7814, lr_0 = 8.4730e-04
Validation auc = 0.727230
Epoch 105
Validation auc = 0.658920
Epoch 106
Validation auc = 0.735915
Epoch 107
Validation auc = 0.657981
Epoch 108
Validation auc = 0.748592
Epoch 109
Loss = 4.9312e+00, PNorm = 73.2766, GNorm = 842.4216, lr_0 = 8.4006e-04
Validation auc = 0.737324
Epoch 110
Validation auc = 0.686150
Epoch 111
Validation auc = 0.727934
Epoch 112
Validation auc = 0.750704
Epoch 113
Validation auc = 0.694601
Epoch 114
Loss = 5.8466e+00, PNorm = 73.4559, GNorm = 646.8201, lr_0 = 8.3288e-04
Validation auc = 0.731221
Epoch 115
Validation auc = 0.722066
Epoch 116
Validation auc = 0.679108
Epoch 117
Validation auc = 0.713615
Epoch 118
Validation auc = 0.707981
Epoch 119
Loss = 5.0084e+00, PNorm = 73.6371, GNorm = 109.9466, lr_0 = 8.2576e-04
Validation auc = 0.678169
Epoch 120
Validation auc = 0.717371
Epoch 121
Validation auc = 0.733099
Epoch 122
Validation auc = 0.713850
Epoch 123
Validation auc = 0.688498
Epoch 124
Loss = 2.7355e+00, PNorm = 73.8500, GNorm = 193.6832, lr_0 = 8.1870e-04
Validation auc = 0.679343
Epoch 125
Validation auc = 0.735681
Epoch 126
Validation auc = 0.726761
Epoch 127
Validation auc = 0.667606
Epoch 128
Validation auc = 0.728638
Epoch 129
Loss = 5.5447e+00, PNorm = 74.0210, GNorm = 751.4337, lr_0 = 8.1170e-04
Validation auc = 0.744601
Epoch 130
Validation auc = 0.679343
Epoch 131
Validation auc = 0.734507
Epoch 132
Validation auc = 0.741315
Epoch 133
Validation auc = 0.695775
Epoch 134
Loss = 6.0963e+00, PNorm = 74.3146, GNorm = 885.9319, lr_0 = 8.0476e-04
Validation auc = 0.704225
Epoch 135
Validation auc = 0.737559
Epoch 136
Validation auc = 0.699296
Epoch 137
Validation auc = 0.753052
Epoch 138
Validation auc = 0.747418
Epoch 139
Loss = 3.6484e+00, PNorm = 74.6999, GNorm = 498.4816, lr_0 = 7.9788e-04
Validation auc = 0.711502
Epoch 140
Validation auc = 0.759390
Epoch 141
Validation auc = 0.757277
Epoch 142
Validation auc = 0.710329
Epoch 143
Validation auc = 0.728169
Epoch 144
Loss = 3.1312e+00, PNorm = 74.9887, GNorm = 262.9579, lr_0 = 7.9106e-04
Validation auc = 0.739202
Epoch 145
Validation auc = 0.674648
Epoch 146
Validation auc = 0.742958
Epoch 147
Validation auc = 0.669014
Epoch 148
Validation auc = 0.708451
Epoch 149
Loss = 2.8350e+00, PNorm = 75.1615, GNorm = 339.1155, lr_0 = 7.8430e-04
Validation auc = 0.758451
Epoch 150
Validation auc = 0.768545
Epoch 151
Validation auc = 0.770892
Epoch 152
Validation auc = 0.787324
Epoch 153
Validation auc = 0.763850
Epoch 154
Loss = 2.9649e+00, PNorm = 75.3870, GNorm = 874.9708, lr_0 = 7.7759e-04
Validation auc = 0.769953
Epoch 155
Validation auc = 0.789671
Epoch 156
Validation auc = 0.723944
Epoch 157
Validation auc = 0.755399
Epoch 158
Validation auc = 0.763615
Epoch 159
Loss = 5.6860e+00, PNorm = 75.7012, GNorm = 768.8411, lr_0 = 7.7095e-04
Validation auc = 0.747887
Epoch 160
Validation auc = 0.705399
Epoch 161
Validation auc = 0.751408
Epoch 162
Validation auc = 0.742488
Epoch 163
Validation auc = 0.673474
Epoch 164
Loss = 6.9915e+00, PNorm = 75.9725, GNorm = 1274.0184, lr_0 = 7.6436e-04
Validation auc = 0.736854
Epoch 165
Validation auc = 0.736620
Epoch 166
Validation auc = 0.692488
Epoch 167
Validation auc = 0.718779
Epoch 168
Validation auc = 0.737793
Epoch 169
Loss = 1.9703e+00, PNorm = 76.2670, GNorm = 198.3392, lr_0 = 7.5782e-04
Validation auc = 0.712676
Epoch 170
Validation auc = 0.727700
Epoch 171
Validation auc = 0.732160
Epoch 172
Validation auc = 0.730047
Epoch 173
Validation auc = 0.749296
Epoch 174
Loss = 2.3778e+00, PNorm = 76.4954, GNorm = 487.2804, lr_0 = 7.5134e-04
Validation auc = 0.731690
Epoch 175
Validation auc = 0.717840
Epoch 176
Validation auc = 0.749765
Epoch 177
Validation auc = 0.726291
Epoch 178
Validation auc = 0.691549
Epoch 179
Loss = 1.3167e+00, PNorm = 76.7706, GNorm = 225.9448, lr_0 = 7.4492e-04
Validation auc = 0.711737
Epoch 180
Validation auc = 0.710329
Epoch 181
Validation auc = 0.683099
Epoch 182
Validation auc = 0.738028
Epoch 183
Validation auc = 0.734272
Epoch 184
Loss = 2.5007e+00, PNorm = 77.0803, GNorm = 415.5214, lr_0 = 7.3855e-04
Validation auc = 0.710094
Epoch 185
Validation auc = 0.720423
Epoch 186
Validation auc = 0.726761
Epoch 187
Validation auc = 0.697183
Epoch 188
Validation auc = 0.714319
Epoch 189
Loss = 2.2427e+00, PNorm = 77.4024, GNorm = 178.8668, lr_0 = 7.3224e-04
Validation auc = 0.712207
Epoch 190
Validation auc = 0.693192
Epoch 191
Validation auc = 0.712676
Epoch 192
Validation auc = 0.729812
Epoch 193
Validation auc = 0.718310
Epoch 194
Loss = 1.7447e+00, PNorm = 77.7331, GNorm = 510.9679, lr_0 = 7.2598e-04
Validation auc = 0.695070
Epoch 195
Validation auc = 0.700939
Epoch 196
Validation auc = 0.719014
Epoch 197
Validation auc = 0.666901
Epoch 198
Validation auc = 0.732394
Epoch 199
Loss = 1.4495e+00, PNorm = 77.9550, GNorm = 354.4354, lr_0 = 7.1977e-04
Validation auc = 0.728169
Epoch 200
Validation auc = 0.690141
Epoch 201
Validation auc = 0.667606
Epoch 202
Validation auc = 0.747653
Epoch 203
Validation auc = 0.739202
Epoch 204
Loss = 2.5102e+00, PNorm = 78.3325, GNorm = 165.8184, lr_0 = 7.1362e-04
Validation auc = 0.687793
Epoch 205
Validation auc = 0.694131
Epoch 206
Validation auc = 0.683099
Epoch 207
Validation auc = 0.712676
Epoch 208
Validation auc = 0.693427
Epoch 209
Loss = 1.5637e+00, PNorm = 78.6713, GNorm = 221.3602, lr_0 = 7.0752e-04
Validation auc = 0.676761
Epoch 210
Validation auc = 0.668545
Epoch 211
Validation auc = 0.686150
Epoch 212
Validation auc = 0.681221
Epoch 213
Validation auc = 0.666667
Epoch 214
Loss = 3.7318e+00, PNorm = 78.9745, GNorm = 262.1635, lr_0 = 7.0147e-04
Validation auc = 0.673005
Epoch 215
Validation auc = 0.685446
Epoch 216
Validation auc = 0.672300
Epoch 217
Validation auc = 0.670657
Epoch 218
Validation auc = 0.708216
Epoch 219
Loss = 2.5289e+00, PNorm = 79.1974, GNorm = 290.5503, lr_0 = 6.9548e-04
Validation auc = 0.694366
Epoch 220
Validation auc = 0.669718
Epoch 221
Validation auc = 0.711268
Epoch 222
Validation auc = 0.707042
Epoch 223
Validation auc = 0.695305
Epoch 224
Loss = 1.7727e+00, PNorm = 79.4792, GNorm = 217.1702, lr_0 = 6.8953e-04
Validation auc = 0.678873
Epoch 225
Validation auc = 0.711268
Epoch 226
Validation auc = 0.715493
Epoch 227
Validation auc = 0.683333
Epoch 228
Validation auc = 0.697653
Epoch 229
Loss = 4.1507e+00, PNorm = 79.6918, GNorm = 329.5778, lr_0 = 6.8364e-04
Validation auc = 0.709624
Epoch 230
Validation auc = 0.683099
Epoch 231
Validation auc = 0.695540
Epoch 232
Validation auc = 0.709859
Epoch 233
Validation auc = 0.707981
Epoch 234
Loss = 2.2094e+00, PNorm = 79.9568, GNorm = 330.8599, lr_0 = 6.7779e-04
Validation auc = 0.658451
Epoch 235
Validation auc = 0.679108
Epoch 236
Validation auc = 0.721362
Epoch 237
Validation auc = 0.732394
Epoch 238
Validation auc = 0.687793
Epoch 239
Loss = 1.6422e+00, PNorm = 80.2350, GNorm = 194.5057, lr_0 = 6.7200e-04
Validation auc = 0.669484
Epoch 240
Validation auc = 0.713380
Epoch 241
Validation auc = 0.748357
Epoch 242
Validation auc = 0.740376
Epoch 243
Validation auc = 0.671362
Epoch 244
Loss = 3.2701e+00, PNorm = 80.4921, GNorm = 580.6565, lr_0 = 6.6625e-04
Validation auc = 0.688732
Epoch 245
Validation auc = 0.708216
Epoch 246
Validation auc = 0.714554
Epoch 247
Validation auc = 0.636150
Epoch 248
Validation auc = 0.713146
Epoch 249
Loss = 3.2469e+00, PNorm = 80.8147, GNorm = 191.3992, lr_0 = 6.6056e-04
Validation auc = 0.675117
Epoch 250
Validation auc = 0.652817
Epoch 251
Validation auc = 0.638732
Epoch 252
Validation auc = 0.667371
Epoch 253
Validation auc = 0.654225
Epoch 254
Loss = 3.6614e-01, PNorm = 81.1943, GNorm = 88.0427, lr_0 = 6.5491e-04
Validation auc = 0.637559
Epoch 255
Validation auc = 0.649296
Epoch 256
Validation auc = 0.682629
Epoch 257
Validation auc = 0.652817
Epoch 258
Validation auc = 0.654225
Epoch 259
Loss = 1.5235e+00, PNorm = 81.5170, GNorm = 344.3188, lr_0 = 6.4931e-04
Validation auc = 0.659859
Epoch 260
Validation auc = 0.670892
Epoch 261
Validation auc = 0.665023
Epoch 262
Validation auc = 0.663380
Epoch 263
Validation auc = 0.660798
Epoch 264
Loss = 1.7420e+00, PNorm = 81.7692, GNorm = 121.0021, lr_0 = 6.4376e-04
Validation auc = 0.712441
Epoch 265
Validation auc = 0.710798
Epoch 266
Validation auc = 0.654460
Epoch 267
Validation auc = 0.695540
Epoch 268
Validation auc = 0.732864
Epoch 269
Loss = 3.3805e-01, PNorm = 82.0287, GNorm = 88.2586, lr_0 = 6.3826e-04
Validation auc = 0.723474
Epoch 270
Validation auc = 0.717840
Epoch 271
Validation auc = 0.683568
Epoch 272
Validation auc = 0.724883
Epoch 273
Validation auc = 0.734507
Epoch 274
Loss = 2.1454e+00, PNorm = 82.2415, GNorm = 543.1463, lr_0 = 6.3280e-04
Validation auc = 0.725822
Epoch 275
Validation auc = 0.661033
Epoch 276
Validation auc = 0.675352
Epoch 277
Validation auc = 0.724883
Epoch 278
Validation auc = 0.727230
Epoch 279
Loss = 1.2060e-01, PNorm = 82.5414, GNorm = 84.9888, lr_0 = 6.2739e-04
Validation auc = 0.682160
Epoch 280
Validation auc = 0.709155
Epoch 281
Validation auc = 0.724648
Epoch 282
Validation auc = 0.690376
Epoch 283
Validation auc = 0.674178
Epoch 284
Loss = 1.6412e+00, PNorm = 82.8387, GNorm = 268.4310, lr_0 = 6.2203e-04
Validation auc = 0.670423
Epoch 285
Validation auc = 0.690141
Epoch 286
Validation auc = 0.686854
Epoch 287
Validation auc = 0.666667
Epoch 288
Validation auc = 0.656103
Epoch 289
Loss = 1.4454e+00, PNorm = 83.0796, GNorm = 94.6614, lr_0 = 6.1671e-04
Validation auc = 0.691549
Epoch 290
Validation auc = 0.680047
Epoch 291
Validation auc = 0.638732
Epoch 292
Validation auc = 0.659859
Epoch 293
Validation auc = 0.671362
Epoch 294
Loss = 3.9723e-01, PNorm = 83.3051, GNorm = 82.4692, lr_0 = 6.1144e-04
Validation auc = 0.672535
Epoch 295
Validation auc = 0.676995
Epoch 296
Validation auc = 0.662207
Epoch 297
Validation auc = 0.657746
Epoch 298
Validation auc = 0.672066
Epoch 299
Loss = 3.3228e-01, PNorm = 83.4426, GNorm = 19.8040, lr_0 = 6.0621e-04
Validation auc = 0.702347
Model 0 best validation auc = 0.789671 on epoch 155
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.805539
Ensemble test auc = 0.805539
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.569484
Epoch 1
Validation auc = 0.497653
Epoch 2
Validation auc = 0.687559
Epoch 3
Validation auc = 0.742019
Epoch 4
Loss = 1.8297e+01, PNorm = 68.3612, GNorm = 1517.1066, lr_0 = 6.5000e-04
Validation auc = 0.593192
Epoch 5
Validation auc = 0.723474
Epoch 6
Validation auc = 0.643662
Epoch 7
Validation auc = 0.705869
Epoch 8
Validation auc = 0.665962
Epoch 9
Loss = 1.9860e+01, PNorm = 68.4988, GNorm = 1841.6713, lr_0 = 9.9743e-04
Validation auc = 0.706338
Epoch 10
Validation auc = 0.697418
Epoch 11
Validation auc = 0.650000
Epoch 12
Validation auc = 0.700235
Epoch 13
Validation auc = 0.733333
Epoch 14
Loss = 1.2361e+01, PNorm = 68.6686, GNorm = 1070.4409, lr_0 = 9.8890e-04
Validation auc = 0.740376
Epoch 15
Validation auc = 0.732394
Epoch 16
Validation auc = 0.596244
Epoch 17
Validation auc = 0.655869
Epoch 18
Validation auc = 0.719484
Epoch 19
Loss = 1.1580e+01, PNorm = 68.8093, GNorm = 471.3275, lr_0 = 9.8045e-04
Validation auc = 0.719014
Epoch 20
Validation auc = 0.715962
Epoch 21
Validation auc = 0.701174
Epoch 22
Validation auc = 0.689202
Epoch 23
Validation auc = 0.704930
Epoch 24
Loss = 9.7480e+00, PNorm = 68.9542, GNorm = 1003.7457, lr_0 = 9.7207e-04
Validation auc = 0.691080
Epoch 25
Validation auc = 0.648826
Epoch 26
Validation auc = 0.600000
Epoch 27
Validation auc = 0.666197
Epoch 28
Validation auc = 0.733568
Epoch 29
Loss = 1.4915e+01, PNorm = 69.0844, GNorm = 1695.2473, lr_0 = 9.6376e-04
Validation auc = 0.725117
Epoch 30
Validation auc = 0.707981
Epoch 31
Validation auc = 0.625352
Epoch 32
Validation auc = 0.716901
Epoch 33
Validation auc = 0.724178
Epoch 34
Loss = 2.0294e+01, PNorm = 69.2292, GNorm = 2079.0200, lr_0 = 9.5552e-04
Validation auc = 0.718310
Epoch 35
Validation auc = 0.726291
Epoch 36
Validation auc = 0.710798
Epoch 37
Validation auc = 0.710329
Epoch 38
Validation auc = 0.671362
Epoch 39
Loss = 8.8474e+00, PNorm = 69.4520, GNorm = 305.9655, lr_0 = 9.4735e-04
Validation auc = 0.634977
Epoch 40
Validation auc = 0.627230
Epoch 41
Validation auc = 0.631455
Epoch 42
Validation auc = 0.635681
Epoch 43
Validation auc = 0.647418
Epoch 44
Loss = 1.0124e+01, PNorm = 69.7149, GNorm = 536.8773, lr_0 = 9.3925e-04
Validation auc = 0.658451
Epoch 45
Validation auc = 0.681221
Epoch 46
Validation auc = 0.694366
Epoch 47
Validation auc = 0.680282
Epoch 48
Validation auc = 0.653756
Epoch 49
Loss = 1.4839e+01, PNorm = 70.1259, GNorm = 298.0444, lr_0 = 9.3122e-04
Validation auc = 0.695540
Epoch 50
Validation auc = 0.712676
Epoch 51
Validation auc = 0.722300
Epoch 52
Validation auc = 0.749061
Epoch 53
Validation auc = 0.665962
Epoch 54
Loss = 1.5770e+01, PNorm = 70.5532, GNorm = 974.5065, lr_0 = 9.2326e-04
Validation auc = 0.664319
Epoch 55
Validation auc = 0.713146
Epoch 56
Validation auc = 0.730986
Epoch 57
Validation auc = 0.720423
Epoch 58
Validation auc = 0.707277
Epoch 59
Loss = 1.6579e+01, PNorm = 71.0060, GNorm = 1570.1694, lr_0 = 9.1537e-04
Validation auc = 0.704460
Epoch 60
Validation auc = 0.672535
Epoch 61
Validation auc = 0.706573
Epoch 62
Validation auc = 0.633568
Epoch 63
Validation auc = 0.710798
Epoch 64
Loss = 8.9062e+00, PNorm = 71.4237, GNorm = 1072.1282, lr_0 = 9.0754e-04
Validation auc = 0.688967
Epoch 65
Validation auc = 0.679812
Epoch 66
Validation auc = 0.720657
Epoch 67
Validation auc = 0.700939
Epoch 68
Validation auc = 0.665023
Epoch 69
Loss = 6.8578e+00, PNorm = 71.7775, GNorm = 847.7431, lr_0 = 8.9978e-04
Validation auc = 0.690376
Epoch 70
Validation auc = 0.647887
Epoch 71
Validation auc = 0.680751
Epoch 72
Validation auc = 0.628873
Epoch 73
Validation auc = 0.627230
Epoch 74
Loss = 9.2128e+00, PNorm = 72.1096, GNorm = 415.9857, lr_0 = 8.9209e-04
Validation auc = 0.689202
Epoch 75
Validation auc = 0.692723
Epoch 76
Validation auc = 0.694366
Epoch 77
Validation auc = 0.650704
Epoch 78
Validation auc = 0.646948
Epoch 79
Loss = 1.0180e+01, PNorm = 72.4377, GNorm = 644.9464, lr_0 = 8.8447e-04
Validation auc = 0.683803
Epoch 80
Validation auc = 0.647183
Epoch 81
Validation auc = 0.650235
Epoch 82
Validation auc = 0.685211
Epoch 83
Validation auc = 0.661268
Epoch 84
Loss = 2.7147e+00, PNorm = 72.6495, GNorm = 429.9753, lr_0 = 8.7691e-04
Validation auc = 0.692958
Epoch 85
Validation auc = 0.674648
Epoch 86
Validation auc = 0.691549
Epoch 87
Validation auc = 0.669718
Epoch 88
Validation auc = 0.645305
Epoch 89
Loss = 3.4099e+00, PNorm = 72.9524, GNorm = 157.3844, lr_0 = 8.6941e-04
Validation auc = 0.684038
Epoch 90
Validation auc = 0.661502
Epoch 91
Validation auc = 0.661033
Epoch 92
Validation auc = 0.701408
Epoch 93
Validation auc = 0.655869
Epoch 94
Loss = 3.5633e+00, PNorm = 73.1014, GNorm = 291.9027, lr_0 = 8.6198e-04
Validation auc = 0.677230
Epoch 95
Validation auc = 0.717371
Epoch 96
Validation auc = 0.650704
Epoch 97
Validation auc = 0.683568
Epoch 98
Validation auc = 0.650000
Epoch 99
Loss = 8.6341e+00, PNorm = 73.4422, GNorm = 882.0410, lr_0 = 8.5461e-04
Validation auc = 0.660798
Epoch 100
Validation auc = 0.698592
Epoch 101
Validation auc = 0.709624
Epoch 102
Validation auc = 0.719014
Epoch 103
Validation auc = 0.730047
Epoch 104
Loss = 3.5003e-01, PNorm = 73.8621, GNorm = 137.2109, lr_0 = 8.4730e-04
Validation auc = 0.732864
Epoch 105
Validation auc = 0.736150
Epoch 106
Validation auc = 0.722535
Epoch 107
Validation auc = 0.722535
Epoch 108
Validation auc = 0.656103
Epoch 109
Loss = 4.9284e+00, PNorm = 74.0714, GNorm = 218.5692, lr_0 = 8.4006e-04
Validation auc = 0.709155
Epoch 110
Validation auc = 0.698122
Epoch 111
Validation auc = 0.678638
Epoch 112
Validation auc = 0.687324
Epoch 113
Validation auc = 0.690610
Epoch 114
Loss = 6.1203e+00, PNorm = 74.3206, GNorm = 379.6381, lr_0 = 8.3288e-04
Validation auc = 0.673239
Epoch 115
Validation auc = 0.701408
Epoch 116
Validation auc = 0.700704
Epoch 117
Validation auc = 0.713380
Epoch 118
Validation auc = 0.734507
Epoch 119
Loss = 8.3486e+00, PNorm = 74.7292, GNorm = 183.7084, lr_0 = 8.2576e-04
Validation auc = 0.694131
Epoch 120
Validation auc = 0.686150
Epoch 121
Validation auc = 0.685446
Epoch 122
Validation auc = 0.747887
Epoch 123
Validation auc = 0.691784
Epoch 124
Loss = 4.8700e+00, PNorm = 75.0618, GNorm = 188.3726, lr_0 = 8.1870e-04
Validation auc = 0.704695
Epoch 125
Validation auc = 0.723239
Epoch 126
Validation auc = 0.687089
Epoch 127
Validation auc = 0.736620
Epoch 128
Validation auc = 0.712911
Epoch 129
Loss = 4.9847e+00, PNorm = 75.5044, GNorm = 587.0426, lr_0 = 8.1170e-04
Validation auc = 0.724178
Epoch 130
Validation auc = 0.740845
Epoch 131
Validation auc = 0.746714
Epoch 132
Validation auc = 0.741784
Epoch 133
Validation auc = 0.675822
Epoch 134
Loss = 5.4400e+00, PNorm = 75.8582, GNorm = 158.2533, lr_0 = 8.0476e-04
Validation auc = 0.678404
Epoch 135
Validation auc = 0.746714
Epoch 136
Validation auc = 0.749296
Epoch 137
Validation auc = 0.745775
Epoch 138
Validation auc = 0.663850
Epoch 139
Loss = 4.7674e+00, PNorm = 76.2235, GNorm = 183.8028, lr_0 = 7.9788e-04
Validation auc = 0.742488
Epoch 140
Validation auc = 0.703286
Epoch 141
Validation auc = 0.684038
Epoch 142
Validation auc = 0.751878
Epoch 143
Validation auc = 0.684977
Epoch 144
Loss = 3.3056e+00, PNorm = 76.5025, GNorm = 1004.2687, lr_0 = 7.9106e-04
Validation auc = 0.764554
Epoch 145
Validation auc = 0.769014
Epoch 146
Validation auc = 0.683099
Epoch 147
Validation auc = 0.747418
Epoch 148
Validation auc = 0.757042
Epoch 149
Loss = 4.7808e+00, PNorm = 76.7769, GNorm = 164.2053, lr_0 = 7.8430e-04
Validation auc = 0.698592
Epoch 150
Validation auc = 0.720423
Epoch 151
Validation auc = 0.704225
Epoch 152
Validation auc = 0.733803
Epoch 153
Validation auc = 0.744836
Epoch 154
Loss = 2.2131e+00, PNorm = 77.0779, GNorm = 175.7536, lr_0 = 7.7759e-04
Validation auc = 0.693192
Epoch 155
Validation auc = 0.755399
Epoch 156
Validation auc = 0.730282
Epoch 157
Validation auc = 0.685681
Epoch 158
Validation auc = 0.706573
Epoch 159
Loss = 2.1906e+00, PNorm = 77.3507, GNorm = 2.0742, lr_0 = 7.7095e-04
Validation auc = 0.690141
Epoch 160
Validation auc = 0.692488
Epoch 161
Validation auc = 0.700939
Epoch 162
Validation auc = 0.721831
Epoch 163
Validation auc = 0.728638
Epoch 164
Loss = 1.1064e+00, PNorm = 77.5712, GNorm = 176.7504, lr_0 = 7.6436e-04
Validation auc = 0.718310
Epoch 165
Validation auc = 0.723474
Epoch 166
Validation auc = 0.746479
Epoch 167
Validation auc = 0.766197
Epoch 168
Validation auc = 0.712207
Epoch 169
Loss = 2.9066e+00, PNorm = 77.8706, GNorm = 135.4013, lr_0 = 7.5782e-04
Validation auc = 0.779577
Epoch 170
Validation auc = 0.785681
Epoch 171
Validation auc = 0.683099
Epoch 172
Validation auc = 0.738263
Epoch 173
Validation auc = 0.733333
Epoch 174
Loss = 7.1278e+00, PNorm = 78.2092, GNorm = 1543.2653, lr_0 = 7.5134e-04
Validation auc = 0.676995
Epoch 175
Validation auc = 0.697887
Epoch 176
Validation auc = 0.742958
Epoch 177
Validation auc = 0.704930
Epoch 178
Validation auc = 0.753991
Epoch 179
Loss = 1.2045e+00, PNorm = 78.6409, GNorm = 134.5047, lr_0 = 7.4492e-04
Validation auc = 0.731221
Epoch 180
Validation auc = 0.727230
Epoch 181
Validation auc = 0.743427
Epoch 182
Validation auc = 0.751174
Epoch 183
Validation auc = 0.739671
Epoch 184
Loss = 3.2213e+00, PNorm = 79.0112, GNorm = 220.7096, lr_0 = 7.3855e-04
Validation auc = 0.683803
Epoch 185
Validation auc = 0.720188
Epoch 186
Validation auc = 0.741315
Epoch 187
Validation auc = 0.711972
Epoch 188
Validation auc = 0.737324
Epoch 189
Loss = 2.6012e+00, PNorm = 79.3032, GNorm = 324.1018, lr_0 = 7.3224e-04
Validation auc = 0.747418
Epoch 190
Validation auc = 0.671127
Epoch 191
Validation auc = 0.673709
Epoch 192
Validation auc = 0.719718
Epoch 193
Validation auc = 0.691549
Epoch 194
Loss = 4.3182e+00, PNorm = 79.5940, GNorm = 796.0141, lr_0 = 7.2598e-04
Validation auc = 0.663850
Epoch 195
Validation auc = 0.713146
Epoch 196
Validation auc = 0.688028
Epoch 197
Validation auc = 0.683568
Epoch 198
Validation auc = 0.690141
Epoch 199
Loss = 4.3571e+00, PNorm = 79.9854, GNorm = 118.9967, lr_0 = 7.1977e-04
Validation auc = 0.650704
Epoch 200
Validation auc = 0.656338
Epoch 201
Validation auc = 0.664085
Epoch 202
Validation auc = 0.645540
Epoch 203
Validation auc = 0.693897
Epoch 204
Loss = 2.1024e+00, PNorm = 80.5803, GNorm = 498.3831, lr_0 = 7.1362e-04
Validation auc = 0.688498
Epoch 205
Validation auc = 0.695305
Epoch 206
Validation auc = 0.695305
Epoch 207
Validation auc = 0.691784
Epoch 208
Validation auc = 0.690610
Epoch 209
Loss = 2.2998e+00, PNorm = 81.0402, GNorm = 279.9728, lr_0 = 7.0752e-04
Validation auc = 0.643662
Epoch 210
Validation auc = 0.688732
Epoch 211
Validation auc = 0.691080
Epoch 212
Validation auc = 0.626761
Epoch 213
Validation auc = 0.663850
Epoch 214
Loss = 1.2628e+00, PNorm = 81.2535, GNorm = 147.6259, lr_0 = 7.0147e-04
Validation auc = 0.673239
Epoch 215
Validation auc = 0.691549
Epoch 216
Validation auc = 0.681455
Epoch 217
Validation auc = 0.688498
Epoch 218
Validation auc = 0.716197
Epoch 219
Loss = 4.1837e+00, PNorm = 81.4693, GNorm = 120.1436, lr_0 = 6.9548e-04
Validation auc = 0.718310
Epoch 220
Validation auc = 0.696714
Epoch 221
Validation auc = 0.749061
Epoch 222
Validation auc = 0.752817
Epoch 223
Validation auc = 0.716197
Epoch 224
Loss = 2.0727e+00, PNorm = 81.7295, GNorm = 464.5729, lr_0 = 6.8953e-04
Validation auc = 0.694601
Epoch 225
Validation auc = 0.746479
Epoch 226
Validation auc = 0.753052
Epoch 227
Validation auc = 0.690610
Epoch 228
Validation auc = 0.705634
Epoch 229
Loss = 2.1268e+00, PNorm = 82.0113, GNorm = 138.6730, lr_0 = 6.8364e-04
Validation auc = 0.733333
Epoch 230
Validation auc = 0.733803
Epoch 231
Validation auc = 0.703052
Epoch 232
Validation auc = 0.729577
Epoch 233
Validation auc = 0.723709
Epoch 234
Loss = 1.0620e+00, PNorm = 82.1964, GNorm = 243.3654, lr_0 = 6.7779e-04
Validation auc = 0.711737
Epoch 235
Validation auc = 0.671831
Epoch 236
Validation auc = 0.712676
Epoch 237
Validation auc = 0.713615
Epoch 238
Validation auc = 0.691549
Epoch 239
Loss = 1.1753e+00, PNorm = 82.3924, GNorm = 567.8361, lr_0 = 6.7200e-04
Validation auc = 0.720423
Epoch 240
Validation auc = 0.726056
Epoch 241
Validation auc = 0.727465
Epoch 242
Validation auc = 0.700235
Epoch 243
Validation auc = 0.716901
Epoch 244
Loss = 1.4865e+00, PNorm = 82.5853, GNorm = 286.2422, lr_0 = 6.6625e-04
Validation auc = 0.739437
Epoch 245
Validation auc = 0.735446
Epoch 246
Validation auc = 0.711972
Epoch 247
Validation auc = 0.710798
Epoch 248
Validation auc = 0.738732
Epoch 249
Loss = 1.4765e+00, PNorm = 82.8068, GNorm = 106.7130, lr_0 = 6.6056e-04
Validation auc = 0.724883
Epoch 250
Validation auc = 0.693897
Epoch 251
Validation auc = 0.692488
Epoch 252
Validation auc = 0.694366
Epoch 253
Validation auc = 0.710563
Epoch 254
Loss = 2.3159e+00, PNorm = 83.0350, GNorm = 87.4207, lr_0 = 6.5491e-04
Validation auc = 0.690610
Epoch 255
Validation auc = 0.687089
Epoch 256
Validation auc = 0.719249
Epoch 257
Validation auc = 0.690845
Epoch 258
Validation auc = 0.720657
Epoch 259
Loss = 4.5372e-02, PNorm = 83.2678, GNorm = 4.7042, lr_0 = 6.4931e-04
Validation auc = 0.719953
Epoch 260
Validation auc = 0.720188
Epoch 261
Validation auc = 0.724178
Epoch 262
Validation auc = 0.710798
Epoch 263
Validation auc = 0.693897
Epoch 264
Loss = 8.7008e-01, PNorm = 83.4406, GNorm = 124.6669, lr_0 = 6.4376e-04
Validation auc = 0.729812
Epoch 265
Validation auc = 0.732864
Epoch 266
Validation auc = 0.696244
Epoch 267
Validation auc = 0.703052
Epoch 268
Validation auc = 0.741549
Epoch 269
Loss = 7.9075e-01, PNorm = 83.6119, GNorm = 142.7586, lr_0 = 6.3826e-04
Validation auc = 0.739906
Epoch 270
Validation auc = 0.745305
Epoch 271
Validation auc = 0.741315
Epoch 272
Validation auc = 0.741080
Epoch 273
Validation auc = 0.707512
Epoch 274
Loss = 7.7872e-01, PNorm = 83.8424, GNorm = 60.7431, lr_0 = 6.3280e-04
Validation auc = 0.751643
Epoch 275
Validation auc = 0.725352
Epoch 276
Validation auc = 0.706103
Epoch 277
Validation auc = 0.734977
Epoch 278
Validation auc = 0.732864
Epoch 279
Loss = 9.7542e-01, PNorm = 84.1807, GNorm = 301.7233, lr_0 = 6.2739e-04
Validation auc = 0.712676
Epoch 280
Validation auc = 0.697418
Epoch 281
Validation auc = 0.707512
Epoch 282
Validation auc = 0.726761
Epoch 283
Validation auc = 0.727230
Epoch 284
Loss = 7.9482e-01, PNorm = 84.4818, GNorm = 191.1551, lr_0 = 6.2203e-04
Validation auc = 0.713380
Epoch 285
Validation auc = 0.676056
Epoch 286
Validation auc = 0.725117
Epoch 287
Validation auc = 0.715258
Epoch 288
Validation auc = 0.712911
Epoch 289
Loss = 1.0061e+00, PNorm = 84.7822, GNorm = 117.0669, lr_0 = 6.1671e-04
Validation auc = 0.684038
Epoch 290
Validation auc = 0.673944
Epoch 291
Validation auc = 0.676761
Epoch 292
Validation auc = 0.706573
Epoch 293
Validation auc = 0.698592
Epoch 294
Loss = 1.0753e+00, PNorm = 85.0697, GNorm = 94.9747, lr_0 = 6.1144e-04
Validation auc = 0.661972
Epoch 295
Validation auc = 0.672300
Epoch 296
Validation auc = 0.688498
Epoch 297
Validation auc = 0.679343
Epoch 298
Validation auc = 0.658920
Epoch 299
Loss = 3.7334e+00, PNorm = 85.2903, GNorm = 212.1599, lr_0 = 6.0621e-04
Validation auc = 0.708920
Model 0 best validation auc = 0.785681 on epoch 170
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.727875
Ensemble test auc = 0.727875
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.590610
Epoch 1
Validation auc = 0.457746
Epoch 2
Validation auc = 0.678873
Epoch 3
Validation auc = 0.690610
Epoch 4
Loss = 2.5876e+01, PNorm = 68.3643, GNorm = 2212.9911, lr_0 = 6.5000e-04
Validation auc = 0.547887
Epoch 5
Validation auc = 0.707981
Epoch 6
Validation auc = 0.676056
Epoch 7
Validation auc = 0.576526
Epoch 8
Validation auc = 0.727700
Epoch 9
Loss = 1.5744e+01, PNorm = 68.4976, GNorm = 413.6931, lr_0 = 9.9743e-04
Validation auc = 0.651174
Epoch 10
Validation auc = 0.670188
Epoch 11
Validation auc = 0.654930
Epoch 12
Validation auc = 0.664789
Epoch 13
Validation auc = 0.637559
Epoch 14
Loss = 2.7013e+01, PNorm = 68.6577, GNorm = 3729.4339, lr_0 = 9.8890e-04
Validation auc = 0.649531
Epoch 15
Validation auc = 0.678873
Epoch 16
Validation auc = 0.735211
Epoch 17
Validation auc = 0.634742
Epoch 18
Validation auc = 0.701174
Epoch 19
Loss = 1.6440e+01, PNorm = 68.8240, GNorm = 410.8736, lr_0 = 9.8045e-04
Validation auc = 0.610329
Epoch 20
Validation auc = 0.704460
Epoch 21
Validation auc = 0.683568
Epoch 22
Validation auc = 0.690610
Epoch 23
Validation auc = 0.690141
Epoch 24
Loss = 1.3369e+01, PNorm = 69.0159, GNorm = 1711.3304, lr_0 = 9.7207e-04
Validation auc = 0.710329
Epoch 25
Validation auc = 0.715258
Epoch 26
Validation auc = 0.726291
Epoch 27
Validation auc = 0.733099
Epoch 28
Validation auc = 0.752817
Epoch 29
Loss = 9.3438e+00, PNorm = 69.1512, GNorm = 458.8331, lr_0 = 9.6376e-04
Validation auc = 0.748122
Epoch 30
Validation auc = 0.759155
Epoch 31
Validation auc = 0.730047
Epoch 32
Validation auc = 0.671362
Epoch 33
Validation auc = 0.676761
Epoch 34
Loss = 8.0825e+00, PNorm = 69.2800, GNorm = 1110.0617, lr_0 = 9.5552e-04
Validation auc = 0.712676
Epoch 35
Validation auc = 0.707042
Epoch 36
Validation auc = 0.577934
Epoch 37
Validation auc = 0.666901
Epoch 38
Validation auc = 0.601408
Epoch 39
Loss = 2.2425e+01, PNorm = 69.4034, GNorm = 426.9631, lr_0 = 9.4735e-04
Validation auc = 0.726761
Epoch 40
Validation auc = 0.693897
Epoch 41
Validation auc = 0.714085
Epoch 42
Validation auc = 0.716432
Epoch 43
Validation auc = 0.663850
Epoch 44
Loss = 1.2703e+01, PNorm = 69.5692, GNorm = 1052.8063, lr_0 = 9.3925e-04
Validation auc = 0.578873
Epoch 45
Validation auc = 0.592958
Epoch 46
Validation auc = 0.695775
Epoch 47
Validation auc = 0.596244
Epoch 48
Validation auc = 0.645070
Epoch 49
Loss = 1.5276e+01, PNorm = 69.7961, GNorm = 1302.4507, lr_0 = 9.3122e-04
Validation auc = 0.636150
Epoch 50
Validation auc = 0.715962
Epoch 51
Validation auc = 0.660798
Epoch 52
Validation auc = 0.631690
Epoch 53
Validation auc = 0.711502
Epoch 54
Loss = 1.1942e+01, PNorm = 70.1725, GNorm = 1082.3938, lr_0 = 9.2326e-04
Validation auc = 0.651174
Epoch 55
Validation auc = 0.631925
Epoch 56
Validation auc = 0.705634
Epoch 57
Validation auc = 0.689437
Epoch 58
Validation auc = 0.628169
Epoch 59
Loss = 1.0477e+01, PNorm = 70.5526, GNorm = 476.1720, lr_0 = 9.1537e-04
Validation auc = 0.729343
Epoch 60
Validation auc = 0.661268
Epoch 61
Validation auc = 0.693897
Epoch 62
Validation auc = 0.727230
Epoch 63
Validation auc = 0.673944
Epoch 64
Loss = 1.2646e+01, PNorm = 70.9375, GNorm = 481.4195, lr_0 = 9.0754e-04
Validation auc = 0.600000
Epoch 65
Validation auc = 0.721831
Epoch 66
Validation auc = 0.647887
Epoch 67
Validation auc = 0.723709
Epoch 68
Validation auc = 0.697183
Epoch 69
Loss = 1.6740e+01, PNorm = 71.2892, GNorm = 477.9291, lr_0 = 8.9978e-04
Validation auc = 0.651643
Epoch 70
Validation auc = 0.726526
Epoch 71
Validation auc = 0.726291
Epoch 72
Validation auc = 0.656103
Epoch 73
Validation auc = 0.682394
Epoch 74
Loss = 1.1548e+01, PNorm = 71.7287, GNorm = 1468.3541, lr_0 = 8.9209e-04
Validation auc = 0.714085
Epoch 75
Validation auc = 0.651878
Epoch 76
Validation auc = 0.727465
Epoch 77
Validation auc = 0.747418
Epoch 78
Validation auc = 0.677934
Epoch 79
Loss = 4.9239e+00, PNorm = 72.0470, GNorm = 497.1273, lr_0 = 8.8447e-04
Validation auc = 0.760798
Epoch 80
Validation auc = 0.745775
Epoch 81
Validation auc = 0.726995
Epoch 82
Validation auc = 0.737089
Epoch 83
Validation auc = 0.701408
Epoch 84
Loss = 5.3907e+00, PNorm = 72.2528, GNorm = 439.6773, lr_0 = 8.7691e-04
Validation auc = 0.684977
Epoch 85
Validation auc = 0.725117
Epoch 86
Validation auc = 0.664789
Epoch 87
Validation auc = 0.697653
Epoch 88
Validation auc = 0.732864
Epoch 89
Loss = 3.7822e+00, PNorm = 72.5275, GNorm = 144.1362, lr_0 = 8.6941e-04
Validation auc = 0.704695
Epoch 90
Validation auc = 0.657512
Epoch 91
Validation auc = 0.728873
Epoch 92
Validation auc = 0.714319
Epoch 93
Validation auc = 0.661737
Epoch 94
Loss = 6.1936e+00, PNorm = 72.6422, GNorm = 523.4572, lr_0 = 8.6198e-04
Validation auc = 0.749296
Epoch 95
Validation auc = 0.676761
Epoch 96
Validation auc = 0.720657
Epoch 97
Validation auc = 0.750704
Epoch 98
Validation auc = 0.607512
Epoch 99
Loss = 1.1603e+01, PNorm = 72.8039, GNorm = 812.6925, lr_0 = 8.5461e-04
Validation auc = 0.666667
Epoch 100
Validation auc = 0.738967
Epoch 101
Validation auc = 0.678169
Epoch 102
Validation auc = 0.751878
Epoch 103
Validation auc = 0.704930
Epoch 104
Loss = 8.9826e+00, PNorm = 72.9319, GNorm = 524.5139, lr_0 = 8.4730e-04
Validation auc = 0.691080
Epoch 105
Validation auc = 0.711033
Epoch 106
Validation auc = 0.682629
Epoch 107
Validation auc = 0.724648
Epoch 108
Validation auc = 0.693192
Epoch 109
Loss = 3.6762e+00, PNorm = 73.1466, GNorm = 528.1614, lr_0 = 8.4006e-04
Validation auc = 0.679812
Epoch 110
Validation auc = 0.672535
Epoch 111
Validation auc = 0.740376
Epoch 112
Validation auc = 0.666901
Epoch 113
Validation auc = 0.707746
Epoch 114
Loss = 1.3110e+01, PNorm = 73.4846, GNorm = 1762.7203, lr_0 = 8.3288e-04
Validation auc = 0.680751
Epoch 115
Validation auc = 0.638028
Epoch 116
Validation auc = 0.718545
Epoch 117
Validation auc = 0.688498
Epoch 118
Validation auc = 0.732629
Epoch 119
Loss = 5.2492e+00, PNorm = 73.7912, GNorm = 940.1408, lr_0 = 8.2576e-04
Validation auc = 0.655869
Epoch 120
Validation auc = 0.720188
Epoch 121
Validation auc = 0.647887
Epoch 122
Validation auc = 0.715728
Epoch 123
Validation auc = 0.649765
Epoch 124
Loss = 6.3939e+00, PNorm = 74.1711, GNorm = 230.7973, lr_0 = 8.1870e-04
Validation auc = 0.710798
Epoch 125
Validation auc = 0.682160
Epoch 126
Validation auc = 0.699765
Epoch 127
Validation auc = 0.705634
Epoch 128
Validation auc = 0.677700
Epoch 129
Loss = 9.2276e+00, PNorm = 74.4015, GNorm = 809.2853, lr_0 = 8.1170e-04
Validation auc = 0.683099
Epoch 130
Validation auc = 0.708451
Epoch 131
Validation auc = 0.707981
Epoch 132
Validation auc = 0.693192
Epoch 133
Validation auc = 0.712676
Epoch 134
Loss = 3.6278e+00, PNorm = 74.5890, GNorm = 681.0021, lr_0 = 8.0476e-04
Validation auc = 0.704225
Epoch 135
Validation auc = 0.728169
Epoch 136
Validation auc = 0.703052
Epoch 137
Validation auc = 0.700235
Epoch 138
Validation auc = 0.702817
Epoch 139
Loss = 2.1156e+00, PNorm = 74.7900, GNorm = 131.0701, lr_0 = 7.9788e-04
Validation auc = 0.700469
Epoch 140
Validation auc = 0.677700
Epoch 141
Validation auc = 0.693897
Epoch 142
Validation auc = 0.741315
Epoch 143
Validation auc = 0.735915
Epoch 144
Loss = 3.3101e+00, PNorm = 75.0255, GNorm = 412.6038, lr_0 = 7.9106e-04
Validation auc = 0.695540
Epoch 145
Validation auc = 0.747887
Epoch 146
Validation auc = 0.753052
Epoch 147
Validation auc = 0.700704
Epoch 148
Validation auc = 0.751643
Epoch 149
Loss = 6.0658e+00, PNorm = 75.2560, GNorm = 530.1320, lr_0 = 7.8430e-04
Validation auc = 0.727934
Epoch 150
Validation auc = 0.684507
Epoch 151
Validation auc = 0.736385
Epoch 152
Validation auc = 0.747887
Epoch 153
Validation auc = 0.682394
Epoch 154
Loss = 5.6630e+00, PNorm = 75.5781, GNorm = 716.8243, lr_0 = 7.7759e-04
Validation auc = 0.741080
Epoch 155
Validation auc = 0.733099
Epoch 156
Validation auc = 0.687324
Epoch 157
Validation auc = 0.728638
Epoch 158
Validation auc = 0.729108
Epoch 159
Loss = 2.3063e+00, PNorm = 75.9397, GNorm = 118.2893, lr_0 = 7.7095e-04
Validation auc = 0.715493
Epoch 160
Validation auc = 0.717371
Epoch 161
Validation auc = 0.727700
Epoch 162
Validation auc = 0.710563
Epoch 163
Validation auc = 0.707512
Epoch 164
Loss = 4.5106e+00, PNorm = 76.2330, GNorm = 511.3765, lr_0 = 7.6436e-04
Validation auc = 0.707042
Epoch 165
Validation auc = 0.710798
Epoch 166
Validation auc = 0.698122
Epoch 167
Validation auc = 0.727934
Epoch 168
Validation auc = 0.728873
Epoch 169
Loss = 1.0921e+00, PNorm = 76.5163, GNorm = 126.8336, lr_0 = 7.5782e-04
Validation auc = 0.721362
Epoch 170
Validation auc = 0.731455
Epoch 171
Validation auc = 0.740141
Epoch 172
Validation auc = 0.694836
Epoch 173
Validation auc = 0.702347
Epoch 174
Loss = 2.9719e+00, PNorm = 76.7424, GNorm = 169.1310, lr_0 = 7.5134e-04
Validation auc = 0.697418
Epoch 175
Validation auc = 0.725352
Epoch 176
Validation auc = 0.730986
Epoch 177
Validation auc = 0.737559
Epoch 178
Validation auc = 0.745305
Epoch 179
Loss = 1.2018e+00, PNorm = 77.0228, GNorm = 215.4042, lr_0 = 7.4492e-04
Validation auc = 0.729577
Epoch 180
Validation auc = 0.734742
Epoch 181
Validation auc = 0.762207
Epoch 182
Validation auc = 0.757981
Epoch 183
Validation auc = 0.692723
Epoch 184
Loss = 2.4859e+00, PNorm = 77.2448, GNorm = 106.3307, lr_0 = 7.3855e-04
Validation auc = 0.753052
Epoch 185
Validation auc = 0.746948
Epoch 186
Validation auc = 0.735211
Epoch 187
Validation auc = 0.732864
Epoch 188
Validation auc = 0.753052
Epoch 189
Loss = 3.0776e+00, PNorm = 77.4860, GNorm = 150.8986, lr_0 = 7.3224e-04
Validation auc = 0.754930
Epoch 190
Validation auc = 0.733803
Epoch 191
Validation auc = 0.734272
Epoch 192
Validation auc = 0.699061
Epoch 193
Validation auc = 0.732394
Epoch 194
Loss = 4.8111e+00, PNorm = 77.6592, GNorm = 538.5428, lr_0 = 7.2598e-04
Validation auc = 0.670657
Epoch 195
Validation auc = 0.693897
Epoch 196
Validation auc = 0.699296
Epoch 197
Validation auc = 0.716901
Epoch 198
Validation auc = 0.705164
Epoch 199
Loss = 6.6456e+00, PNorm = 78.0994, GNorm = 644.9502, lr_0 = 7.1977e-04
Validation auc = 0.732394
Epoch 200
Validation auc = 0.653521
Epoch 201
Validation auc = 0.757042
Epoch 202
Validation auc = 0.680282
Epoch 203
Validation auc = 0.748826
Epoch 204
Loss = 4.5347e+00, PNorm = 78.7688, GNorm = 765.3059, lr_0 = 7.1362e-04
Validation auc = 0.650235
Epoch 205
Validation auc = 0.747418
Epoch 206
Validation auc = 0.703991
Epoch 207
Validation auc = 0.727700
Epoch 208
Validation auc = 0.740845
Epoch 209
Loss = 3.1151e+00, PNorm = 79.2936, GNorm = 430.5443, lr_0 = 7.0752e-04
Validation auc = 0.739437
Epoch 210
Validation auc = 0.728638
Epoch 211
Validation auc = 0.716197
Epoch 212
Validation auc = 0.705869
Epoch 213
Validation auc = 0.704460
Epoch 214
Loss = 3.2407e+00, PNorm = 79.5725, GNorm = 325.8873, lr_0 = 7.0147e-04
Validation auc = 0.705869
Epoch 215
Validation auc = 0.705164
Epoch 216
Validation auc = 0.711268
Epoch 217
Validation auc = 0.727934
Epoch 218
Validation auc = 0.721362
Epoch 219
Loss = 4.1708e+00, PNorm = 79.8351, GNorm = 1067.5847, lr_0 = 6.9548e-04
Validation auc = 0.717840
Epoch 220
Validation auc = 0.737089
Epoch 221
Validation auc = 0.725587
Epoch 222
Validation auc = 0.704695
Epoch 223
Validation auc = 0.716667
Epoch 224
Loss = 3.5344e+00, PNorm = 80.0356, GNorm = 552.8086, lr_0 = 6.8953e-04
Validation auc = 0.728404
Epoch 225
Validation auc = 0.726056
Epoch 226
Validation auc = 0.700704
Epoch 227
Validation auc = 0.697418
Epoch 228
Validation auc = 0.722535
Epoch 229
Loss = 3.3474e+00, PNorm = 80.3184, GNorm = 514.6057, lr_0 = 6.8364e-04
Validation auc = 0.736620
Epoch 230
Validation auc = 0.678404
Epoch 231
Validation auc = 0.700000
Epoch 232
Validation auc = 0.733099
Epoch 233
Validation auc = 0.721127
Epoch 234
Loss = 5.0759e+00, PNorm = 80.6572, GNorm = 608.9526, lr_0 = 6.7779e-04
Validation auc = 0.684272
Epoch 235
Validation auc = 0.701878
Epoch 236
Validation auc = 0.699296
Epoch 237
Validation auc = 0.701643
Epoch 238
Validation auc = 0.709624
Epoch 239
Loss = 2.3641e+00, PNorm = 80.9148, GNorm = 135.0498, lr_0 = 6.7200e-04
Validation auc = 0.704225
Epoch 240
Validation auc = 0.719249
Epoch 241
Validation auc = 0.724883
Epoch 242
Validation auc = 0.720892
Epoch 243
Validation auc = 0.717371
Epoch 244
Loss = 1.2804e+00, PNorm = 80.9864, GNorm = 230.7425, lr_0 = 6.6625e-04
Validation auc = 0.715258
Epoch 245
Validation auc = 0.704460
Epoch 246
Validation auc = 0.725587
Epoch 247
Validation auc = 0.741080
Epoch 248
Validation auc = 0.732160
Epoch 249
Loss = 2.7171e+00, PNorm = 81.1488, GNorm = 438.6952, lr_0 = 6.6056e-04
Validation auc = 0.713850
Epoch 250
Validation auc = 0.751643
Epoch 251
Validation auc = 0.764319
Epoch 252
Validation auc = 0.762911
Epoch 253
Validation auc = 0.740845
Epoch 254
Loss = 1.1187e+00, PNorm = 81.2600, GNorm = 157.5979, lr_0 = 6.5491e-04
Validation auc = 0.759624
Epoch 255
Validation auc = 0.773709
Epoch 256
Validation auc = 0.755869
Epoch 257
Validation auc = 0.761972
Epoch 258
Validation auc = 0.739906
Epoch 259
Loss = 2.9943e+00, PNorm = 81.3774, GNorm = 989.6871, lr_0 = 6.4931e-04
Validation auc = 0.718075
Epoch 260
Validation auc = 0.760094
Epoch 261
Validation auc = 0.756808
Epoch 262
Validation auc = 0.694836
Epoch 263
Validation auc = 0.718779
Epoch 264
Loss = 6.2569e-01, PNorm = 81.5935, GNorm = 96.3832, lr_0 = 6.4376e-04
Validation auc = 0.745305
Epoch 265
Validation auc = 0.736620
Epoch 266
Validation auc = 0.684507
Epoch 267
Validation auc = 0.712911
Epoch 268
Validation auc = 0.723944
Epoch 269
Loss = 3.9309e+00, PNorm = 81.8878, GNorm = 528.4775, lr_0 = 6.3826e-04
Validation auc = 0.712207
Epoch 270
Validation auc = 0.696714
Epoch 271
Validation auc = 0.707277
Epoch 272
Validation auc = 0.719249
Epoch 273
Validation auc = 0.713850
Epoch 274
Loss = 1.2638e+00, PNorm = 82.2159, GNorm = 199.5610, lr_0 = 6.3280e-04
Validation auc = 0.695775
Epoch 275
Validation auc = 0.704695
Epoch 276
Validation auc = 0.723239
Epoch 277
Validation auc = 0.707277
Epoch 278
Validation auc = 0.705869
Epoch 279
Loss = 1.5283e+00, PNorm = 82.5823, GNorm = 394.2496, lr_0 = 6.2739e-04
Validation auc = 0.694836
Epoch 280
Validation auc = 0.700000
Epoch 281
Validation auc = 0.717606
Epoch 282
Validation auc = 0.723005
Epoch 283
Validation auc = 0.706338
Epoch 284
Loss = 2.6327e+00, PNorm = 82.8579, GNorm = 371.8576, lr_0 = 6.2203e-04
Validation auc = 0.699765
Epoch 285
Validation auc = 0.712676
Epoch 286
Validation auc = 0.743192
Epoch 287
Validation auc = 0.723709
Epoch 288
Validation auc = 0.711268
Epoch 289
Loss = 7.6259e-01, PNorm = 83.0696, GNorm = 153.6028, lr_0 = 6.1671e-04
Validation auc = 0.734507
Epoch 290
Validation auc = 0.726995
Epoch 291
Validation auc = 0.698592
Epoch 292
Validation auc = 0.693427
Epoch 293
Validation auc = 0.696948
Epoch 294
Loss = 2.3583e+00, PNorm = 83.3312, GNorm = 295.0690, lr_0 = 6.1144e-04
Validation auc = 0.704930
Epoch 295
Validation auc = 0.686150
Epoch 296
Validation auc = 0.689906
Epoch 297
Validation auc = 0.694836
Epoch 298
Validation auc = 0.710329
Epoch 299
Loss = 6.9728e-01, PNorm = 83.6232, GNorm = 36.5910, lr_0 = 6.0621e-04
Validation auc = 0.708685
Model 0 best validation auc = 0.773709 on epoch 255
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.712523
Ensemble test auc = 0.712523
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.608451
Epoch 1
Validation auc = 0.725822
Epoch 2
Validation auc = 0.684742
Epoch 3
Validation auc = 0.702817
Epoch 4
Loss = 2.3450e+01, PNorm = 68.3690, GNorm = 990.5795, lr_0 = 6.5000e-04
Validation auc = 0.590141
Epoch 5
Validation auc = 0.750235
Epoch 6
Validation auc = 0.714085
Epoch 7
Validation auc = 0.685211
Epoch 8
Validation auc = 0.653991
Epoch 9
Loss = 3.6803e+01, PNorm = 68.5045, GNorm = 3134.3748, lr_0 = 9.9743e-04
Validation auc = 0.591315
Epoch 10
Validation auc = 0.676995
Epoch 11
Validation auc = 0.555164
Epoch 12
Validation auc = 0.719718
Epoch 13
Validation auc = 0.678169
Epoch 14
Loss = 2.6442e+01, PNorm = 68.6570, GNorm = 1593.4134, lr_0 = 9.8890e-04
Validation auc = 0.546009
Epoch 15
Validation auc = 0.683803
Epoch 16
Validation auc = 0.671362
Epoch 17
Validation auc = 0.537324
Epoch 18
Validation auc = 0.697183
Epoch 19
Loss = 2.2541e+01, PNorm = 68.7836, GNorm = 1886.4238, lr_0 = 9.8045e-04
Validation auc = 0.672066
Epoch 20
Validation auc = 0.659390
Epoch 21
Validation auc = 0.691784
Epoch 22
Validation auc = 0.634742
Epoch 23
Validation auc = 0.566197
Epoch 24
Loss = 2.0559e+01, PNorm = 68.8972, GNorm = 1638.9281, lr_0 = 9.7207e-04
Validation auc = 0.658920
Epoch 25
Validation auc = 0.634742
Epoch 26
Validation auc = 0.538967
Epoch 27
Validation auc = 0.634742
Epoch 28
Validation auc = 0.628169
Epoch 29
Loss = 1.1594e+01, PNorm = 69.0267, GNorm = 1195.6438, lr_0 = 9.6376e-04
Validation auc = 0.513380
Epoch 30
Validation auc = 0.626291
Epoch 31
Validation auc = 0.672770
Epoch 32
Validation auc = 0.664085
Epoch 33
Validation auc = 0.656338
Epoch 34
Loss = 1.8149e+01, PNorm = 69.1604, GNorm = 1345.8038, lr_0 = 9.5552e-04
Validation auc = 0.535681
Epoch 35
Validation auc = 0.668310
Epoch 36
Validation auc = 0.694836
Epoch 37
Validation auc = 0.536385
Epoch 38
Validation auc = 0.716667
Epoch 39
Loss = 1.5759e+01, PNorm = 69.3771, GNorm = 2322.8591, lr_0 = 9.4735e-04
Validation auc = 0.716197
Epoch 40
Validation auc = 0.572066
Epoch 41
Validation auc = 0.719718
Epoch 42
Validation auc = 0.633568
Epoch 43
Validation auc = 0.642254
Epoch 44
Loss = 1.8363e+01, PNorm = 69.6203, GNorm = 934.0903, lr_0 = 9.3925e-04
Validation auc = 0.676995
Epoch 45
Validation auc = 0.657746
Epoch 46
Validation auc = 0.651174
Epoch 47
Validation auc = 0.652113
Epoch 48
Validation auc = 0.650235
Epoch 49
Loss = 6.2096e+00, PNorm = 69.8576, GNorm = 579.6376, lr_0 = 9.3122e-04
Validation auc = 0.633333
Epoch 50
Validation auc = 0.655634
Epoch 51
Validation auc = 0.638498
Epoch 52
Validation auc = 0.626995
Epoch 53
Validation auc = 0.653521
Epoch 54
Loss = 5.3228e+00, PNorm = 70.0334, GNorm = 206.5283, lr_0 = 9.2326e-04
Validation auc = 0.625822
Epoch 55
Validation auc = 0.632864
Epoch 56
Validation auc = 0.608451
Epoch 57
Validation auc = 0.606573
Epoch 58
Validation auc = 0.644131
Epoch 59
Loss = 7.6999e+00, PNorm = 70.2962, GNorm = 337.2575, lr_0 = 9.1537e-04
Validation auc = 0.624413
Epoch 60
Validation auc = 0.650000
Epoch 61
Validation auc = 0.632160
Epoch 62
Validation auc = 0.709859
Epoch 63
Validation auc = 0.642254
Epoch 64
Loss = 5.7884e+00, PNorm = 70.6968, GNorm = 230.3239, lr_0 = 9.0754e-04
Validation auc = 0.706573
Epoch 65
Validation auc = 0.707512
Epoch 66
Validation auc = 0.650235
Epoch 67
Validation auc = 0.715258
Epoch 68
Validation auc = 0.649765
Epoch 69
Loss = 7.8466e+00, PNorm = 71.1131, GNorm = 208.9026, lr_0 = 8.9978e-04
Validation auc = 0.647653
Epoch 70
Validation auc = 0.727934
Epoch 71
Validation auc = 0.734742
Epoch 72
Validation auc = 0.706338
Epoch 73
Validation auc = 0.727934
Epoch 74
Loss = 2.9434e+00, PNorm = 71.3762, GNorm = 175.3366, lr_0 = 8.9209e-04
Validation auc = 0.718545
Epoch 75
Validation auc = 0.654695
Epoch 76
Validation auc = 0.704695
Epoch 77
Validation auc = 0.706103
Epoch 78
Validation auc = 0.748826
Epoch 79
Loss = 4.2017e+00, PNorm = 71.6155, GNorm = 232.8337, lr_0 = 8.8447e-04
Validation auc = 0.721831
Epoch 80
Validation auc = 0.726056
Epoch 81
Validation auc = 0.750000
Epoch 82
Validation auc = 0.712207
Epoch 83
Validation auc = 0.645305
Epoch 84
Loss = 4.0267e+00, PNorm = 71.8315, GNorm = 183.1308, lr_0 = 8.7691e-04
Validation auc = 0.681925
Epoch 85
Validation auc = 0.653756
Epoch 86
Validation auc = 0.735681
Epoch 87
Validation auc = 0.719014
Epoch 88
Validation auc = 0.706103
Epoch 89
Loss = 5.2323e+00, PNorm = 72.0795, GNorm = 784.4375, lr_0 = 8.6941e-04
Validation auc = 0.760798
Epoch 90
Validation auc = 0.738967
Epoch 91
Validation auc = 0.679343
Epoch 92
Validation auc = 0.751878
Epoch 93
Validation auc = 0.748357
Epoch 94
Loss = 4.2592e+00, PNorm = 72.4269, GNorm = 960.5948, lr_0 = 8.6198e-04
Validation auc = 0.652347
Epoch 95
Validation auc = 0.756338
Epoch 96
Validation auc = 0.729577
Epoch 97
Validation auc = 0.658216
Epoch 98
Validation auc = 0.692254
Epoch 99
Loss = 1.2753e+01, PNorm = 72.7775, GNorm = 1193.5551, lr_0 = 8.5461e-04
Validation auc = 0.736385
Epoch 100
Validation auc = 0.676526
Epoch 101
Validation auc = 0.749296
Epoch 102
Validation auc = 0.758685
Epoch 103
Validation auc = 0.696479
Epoch 104
Loss = 2.4200e+00, PNorm = 73.0321, GNorm = 117.3028, lr_0 = 8.4730e-04
Validation auc = 0.723709
Epoch 105
Validation auc = 0.748592
Epoch 106
Validation auc = 0.679812
Epoch 107
Validation auc = 0.652347
Epoch 108
Validation auc = 0.690610
Epoch 109
Loss = 5.7571e+00, PNorm = 73.2799, GNorm = 966.1132, lr_0 = 8.4006e-04
Validation auc = 0.651408
Epoch 110
Validation auc = 0.707981
Epoch 111
Validation auc = 0.681690
Epoch 112
Validation auc = 0.648122
Epoch 113
Validation auc = 0.691549
Epoch 114
Loss = 2.3754e+00, PNorm = 73.5606, GNorm = 100.4329, lr_0 = 8.3288e-04
Validation auc = 0.672535
Epoch 115
Validation auc = 0.695305
Epoch 116
Validation auc = 0.712676
Epoch 117
Validation auc = 0.700000
Epoch 118
Validation auc = 0.706573
Epoch 119
Loss = 3.5509e+00, PNorm = 73.8039, GNorm = 177.1271, lr_0 = 8.2576e-04
Validation auc = 0.706573
Epoch 120
Validation auc = 0.701878
Epoch 121
Validation auc = 0.734038
Epoch 122
Validation auc = 0.658451
Epoch 123
Validation auc = 0.757277
Epoch 124
Loss = 4.1322e+00, PNorm = 74.1243, GNorm = 666.5957, lr_0 = 8.1870e-04
Validation auc = 0.748357
Epoch 125
Validation auc = 0.685211
Epoch 126
Validation auc = 0.761737
Epoch 127
Validation auc = 0.710094
Epoch 128
Validation auc = 0.718545
Epoch 129
Loss = 2.8904e+00, PNorm = 74.3572, GNorm = 447.3478, lr_0 = 8.1170e-04
Validation auc = 0.668779
Epoch 130
Validation auc = 0.767606
Epoch 131
Validation auc = 0.668545
Epoch 132
Validation auc = 0.666901
Epoch 133
Validation auc = 0.736385
Epoch 134
Loss = 3.0200e+00, PNorm = 74.5624, GNorm = 430.7928, lr_0 = 8.0476e-04
Validation auc = 0.722300
Epoch 135
Validation auc = 0.751643
Epoch 136
Validation auc = 0.723709
Epoch 137
Validation auc = 0.672770
Epoch 138
Validation auc = 0.716901
Epoch 139
Loss = 4.8927e+00, PNorm = 74.8562, GNorm = 259.1710, lr_0 = 7.9788e-04
Validation auc = 0.720188
Epoch 140
Validation auc = 0.706103
Epoch 141
Validation auc = 0.732394
Epoch 142
Validation auc = 0.715493
Epoch 143
Validation auc = 0.727230
Epoch 144
Loss = 7.7594e+00, PNorm = 75.1908, GNorm = 591.9699, lr_0 = 7.9106e-04
Validation auc = 0.727230
Epoch 145
Validation auc = 0.701878
Epoch 146
Validation auc = 0.688732
Epoch 147
Validation auc = 0.669249
Epoch 148
Validation auc = 0.720423
Epoch 149
Loss = 5.0261e+00, PNorm = 75.4958, GNorm = 1053.9531, lr_0 = 7.8430e-04
Validation auc = 0.664554
Epoch 150
Validation auc = 0.690610
Epoch 151
Validation auc = 0.708451
Epoch 152
Validation auc = 0.652582
Epoch 153
Validation auc = 0.738967
Epoch 154
Loss = 4.0065e+00, PNorm = 75.8102, GNorm = 781.9479, lr_0 = 7.7759e-04
Validation auc = 0.749296
Epoch 155
Validation auc = 0.719249
Epoch 156
Validation auc = 0.753286
Epoch 157
Validation auc = 0.719953
Epoch 158
Validation auc = 0.763380
Epoch 159
Loss = 1.3575e+00, PNorm = 76.0889, GNorm = 140.0445, lr_0 = 7.7095e-04
Validation auc = 0.764319
Epoch 160
Validation auc = 0.775587
Epoch 161
Validation auc = 0.700235
Epoch 162
Validation auc = 0.782160
Epoch 163
Validation auc = 0.763615
Epoch 164
Loss = 4.2831e+00, PNorm = 76.4539, GNorm = 637.1519, lr_0 = 7.6436e-04
Validation auc = 0.684507
Epoch 165
Validation auc = 0.747887
Epoch 166
Validation auc = 0.749765
Epoch 167
Validation auc = 0.680516
Epoch 168
Validation auc = 0.738263
Epoch 169
Loss = 4.6382e+00, PNorm = 76.8397, GNorm = 183.8959, lr_0 = 7.5782e-04
Validation auc = 0.733333
Epoch 170
Validation auc = 0.703756
Epoch 171
Validation auc = 0.694366
Epoch 172
Validation auc = 0.727230
Epoch 173
Validation auc = 0.707042
Epoch 174
Loss = 3.5988e+00, PNorm = 77.1342, GNorm = 148.1742, lr_0 = 7.5134e-04
Validation auc = 0.687559
Epoch 175
Validation auc = 0.710094
Epoch 176
Validation auc = 0.739437
Epoch 177
Validation auc = 0.658451
Epoch 178
Validation auc = 0.715493
Epoch 179
Loss = 2.3345e+00, PNorm = 77.4993, GNorm = 608.6812, lr_0 = 7.4492e-04
Validation auc = 0.714789
Epoch 180
Validation auc = 0.694366
Epoch 181
Validation auc = 0.656573
Epoch 182
Validation auc = 0.702347
Epoch 183
Validation auc = 0.700939
Epoch 184
Loss = 2.1944e+00, PNorm = 77.8612, GNorm = 174.9704, lr_0 = 7.3855e-04
Validation auc = 0.646714
Epoch 185
Validation auc = 0.670892
Epoch 186
Validation auc = 0.699061
Epoch 187
Validation auc = 0.672535
Epoch 188
Validation auc = 0.687324
Epoch 189
Loss = 3.7176e+00, PNorm = 78.3289, GNorm = 143.5602, lr_0 = 7.3224e-04
Validation auc = 0.729577
Epoch 190
Validation auc = 0.745070
Epoch 191
Validation auc = 0.679343
Epoch 192
Validation auc = 0.695540
Epoch 193
Validation auc = 0.687793
Epoch 194
Loss = 9.5294e+00, PNorm = 78.7874, GNorm = 822.6081, lr_0 = 7.2598e-04
Validation auc = 0.751174
Epoch 195
Validation auc = 0.694836
Epoch 196
Validation auc = 0.746009
Epoch 197
Validation auc = 0.749531
Epoch 198
Validation auc = 0.676995
Epoch 199
Loss = 3.7692e+00, PNorm = 79.2899, GNorm = 610.4817, lr_0 = 7.1977e-04
Validation auc = 0.669014
Epoch 200
Validation auc = 0.703756
Epoch 201
Validation auc = 0.685446
Epoch 202
Validation auc = 0.666667
Epoch 203
Validation auc = 0.676526
Epoch 204
Loss = 1.2189e+00, PNorm = 79.6719, GNorm = 81.2621, lr_0 = 7.1362e-04
Validation auc = 0.694836
Epoch 205
Validation auc = 0.707981
Epoch 206
Validation auc = 0.720657
Epoch 207
Validation auc = 0.768310
Epoch 208
Validation auc = 0.688732
Epoch 209
Loss = 1.5105e+00, PNorm = 79.8985, GNorm = 471.8053, lr_0 = 7.0752e-04
Validation auc = 0.753521
Epoch 210
Validation auc = 0.742958
Epoch 211
Validation auc = 0.689906
Epoch 212
Validation auc = 0.702817
Epoch 213
Validation auc = 0.696714
Epoch 214
Loss = 1.5277e+00, PNorm = 80.1818, GNorm = 276.5956, lr_0 = 7.0147e-04
Validation auc = 0.693897
Epoch 215
Validation auc = 0.686150
Epoch 216
Validation auc = 0.723474
Epoch 217
Validation auc = 0.725822
Epoch 218
Validation auc = 0.677934
Epoch 219
Loss = 2.4592e+00, PNorm = 80.5207, GNorm = 278.5170, lr_0 = 6.9548e-04
Validation auc = 0.666197
Epoch 220
Validation auc = 0.738967
Epoch 221
Validation auc = 0.751408
Epoch 222
Validation auc = 0.661033
Epoch 223
Validation auc = 0.714085
Epoch 224
Loss = 3.4256e+00, PNorm = 80.9992, GNorm = 717.5070, lr_0 = 6.8953e-04
Validation auc = 0.753286
Epoch 225
Validation auc = 0.704225
Epoch 226
Validation auc = 0.689906
Epoch 227
Validation auc = 0.741315
Epoch 228
Validation auc = 0.674648
Epoch 229
Loss = 8.7629e+00, PNorm = 81.5360, GNorm = 1224.6849, lr_0 = 6.8364e-04
Validation auc = 0.732864
Epoch 230
Validation auc = 0.679343
Epoch 231
Validation auc = 0.737324
Epoch 232
Validation auc = 0.716432
Epoch 233
Validation auc = 0.747183
Epoch 234
Loss = 4.5723e+00, PNorm = 82.0997, GNorm = 189.3270, lr_0 = 6.7779e-04
Validation auc = 0.703286
Epoch 235
Validation auc = 0.728638
Epoch 236
Validation auc = 0.705164
Epoch 237
Validation auc = 0.749061
Epoch 238
Validation auc = 0.700939
Epoch 239
Loss = 4.0514e+00, PNorm = 82.5839, GNorm = 782.8039, lr_0 = 6.7200e-04
Validation auc = 0.683568
Epoch 240
Validation auc = 0.742723
Epoch 241
Validation auc = 0.727230
Epoch 242
Validation auc = 0.720657
Epoch 243
Validation auc = 0.692019
Epoch 244
Loss = 2.8309e+00, PNorm = 82.9984, GNorm = 84.8007, lr_0 = 6.6625e-04
Validation auc = 0.725117
Epoch 245
Validation auc = 0.730516
Epoch 246
Validation auc = 0.714554
Epoch 247
Validation auc = 0.673709
Epoch 248
Validation auc = 0.736150
Epoch 249
Loss = 1.6778e+00, PNorm = 83.2834, GNorm = 663.0868, lr_0 = 6.6056e-04
Validation auc = 0.731925
Epoch 250
Validation auc = 0.702817
Epoch 251
Validation auc = 0.680282
Epoch 252
Validation auc = 0.727934
Epoch 253
Validation auc = 0.722770
Epoch 254
Loss = 2.0091e+00, PNorm = 83.6132, GNorm = 246.5533, lr_0 = 6.5491e-04
Validation auc = 0.722066
Epoch 255
Validation auc = 0.701408
Epoch 256
Validation auc = 0.685211
Epoch 257
Validation auc = 0.690141
Epoch 258
Validation auc = 0.721127
Epoch 259
Loss = 5.4014e-01, PNorm = 83.9917, GNorm = 242.5329, lr_0 = 6.4931e-04
Validation auc = 0.741549
Epoch 260
Validation auc = 0.742254
Epoch 261
Validation auc = 0.705164
Epoch 262
Validation auc = 0.653052
Epoch 263
Validation auc = 0.678873
Epoch 264
Loss = 1.7351e-01, PNorm = 84.2961, GNorm = 0.6221, lr_0 = 6.4376e-04
Validation auc = 0.706103
Epoch 265
Validation auc = 0.714085
Epoch 266
Validation auc = 0.685681
Epoch 267
Validation auc = 0.664789
Epoch 268
Validation auc = 0.671127
Epoch 269
Loss = 1.1372e+00, PNorm = 84.5535, GNorm = 315.9927, lr_0 = 6.3826e-04
Validation auc = 0.668545
Epoch 270
Validation auc = 0.698122
Epoch 271
Validation auc = 0.708216
Epoch 272
Validation auc = 0.673709
Epoch 273
Validation auc = 0.690141
Epoch 274
Loss = 2.6665e+00, PNorm = 84.7750, GNorm = 132.6072, lr_0 = 6.3280e-04
Validation auc = 0.721831
Epoch 275
Validation auc = 0.724883
Epoch 276
Validation auc = 0.693427
Epoch 277
Validation auc = 0.690610
Epoch 278
Validation auc = 0.712207
Epoch 279
Loss = 3.3303e+00, PNorm = 85.1011, GNorm = 371.3531, lr_0 = 6.2739e-04
Validation auc = 0.699061
Epoch 280
Validation auc = 0.677465
Epoch 281
Validation auc = 0.680751
Epoch 282
Validation auc = 0.703756
Epoch 283
Validation auc = 0.711033
Epoch 284
Loss = 1.9748e+00, PNorm = 85.4210, GNorm = 305.3429, lr_0 = 6.2203e-04
Validation auc = 0.707981
Epoch 285
Validation auc = 0.696714
Epoch 286
Validation auc = 0.708920
Epoch 287
Validation auc = 0.719014
Epoch 288
Validation auc = 0.717136
Epoch 289
Loss = 6.4467e-01, PNorm = 85.6738, GNorm = 133.2367, lr_0 = 6.1671e-04
Validation auc = 0.701408
Epoch 290
Validation auc = 0.696714
Epoch 291
Validation auc = 0.700000
Epoch 292
Validation auc = 0.706103
Epoch 293
Validation auc = 0.707042
Epoch 294
Loss = 4.7027e-01, PNorm = 85.8982, GNorm = 296.3404, lr_0 = 6.1144e-04
Validation auc = 0.698122
Epoch 295
Validation auc = 0.717136
Epoch 296
Validation auc = 0.724883
Epoch 297
Validation auc = 0.699531
Epoch 298
Validation auc = 0.703286
Epoch 299
Loss = 5.9773e-01, PNorm = 86.1697, GNorm = 71.1232, lr_0 = 6.0621e-04
Validation auc = 0.701878
Model 0 best validation auc = 0.782160 on epoch 162
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.831427
Ensemble test auc = 0.831427
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.582864
Epoch 1
Validation auc = 0.691080
Epoch 2
Validation auc = 0.700000
Epoch 3
Validation auc = 0.791080
Epoch 4
Loss = 2.5970e+01, PNorm = 68.3689, GNorm = 1596.1936, lr_0 = 6.5000e-04
Validation auc = 0.775587
Epoch 5
Validation auc = 0.618310
Epoch 6
Validation auc = 0.685915
Epoch 7
Validation auc = 0.709390
Epoch 8
Validation auc = 0.698122
Epoch 9
Loss = 1.4990e+01, PNorm = 68.5086, GNorm = 457.6808, lr_0 = 9.9743e-04
Validation auc = 0.679812
Epoch 10
Validation auc = 0.715962
Epoch 11
Validation auc = 0.631455
Epoch 12
Validation auc = 0.630282
Epoch 13
Validation auc = 0.609859
Epoch 14
Loss = 1.1208e+01, PNorm = 68.6667, GNorm = 248.9632, lr_0 = 9.8890e-04
Validation auc = 0.674178
Epoch 15
Validation auc = 0.678404
Epoch 16
Validation auc = 0.656808
Epoch 17
Validation auc = 0.689202
Epoch 18
Validation auc = 0.723474
Epoch 19
Loss = 7.8446e+00, PNorm = 68.8083, GNorm = 1367.8949, lr_0 = 9.8045e-04
Validation auc = 0.715728
Epoch 20
Validation auc = 0.669718
Epoch 21
Validation auc = 0.625352
Epoch 22
Validation auc = 0.609859
Epoch 23
Validation auc = 0.604225
Epoch 24
Loss = 2.6625e+01, PNorm = 68.9444, GNorm = 2059.1838, lr_0 = 9.7207e-04
Validation auc = 0.569014
Epoch 25
Validation auc = 0.602817
Epoch 26
Validation auc = 0.648357
Epoch 27
Validation auc = 0.686385
Epoch 28
Validation auc = 0.613380
Epoch 29
Loss = 2.1183e+01, PNorm = 69.1564, GNorm = 828.4987, lr_0 = 9.6376e-04
Validation auc = 0.615962
Epoch 30
Validation auc = 0.692019
Epoch 31
Validation auc = 0.660563
Epoch 32
Validation auc = 0.676291
Epoch 33
Validation auc = 0.652817
Epoch 34
Loss = 1.9811e+01, PNorm = 69.4954, GNorm = 1898.1635, lr_0 = 9.5552e-04
Validation auc = 0.712676
Epoch 35
Validation auc = 0.677700
Epoch 36
Validation auc = 0.638967
Epoch 37
Validation auc = 0.695305
Epoch 38
Validation auc = 0.609390
Epoch 39
Loss = 1.1425e+01, PNorm = 69.7625, GNorm = 209.3158, lr_0 = 9.4735e-04
Validation auc = 0.613146
Epoch 40
Validation auc = 0.722066
Epoch 41
Validation auc = 0.696714
Epoch 42
Validation auc = 0.674648
Epoch 43
Validation auc = 0.730986
Epoch 44
Loss = 9.4623e+00, PNorm = 69.9914, GNorm = 291.6047, lr_0 = 9.3925e-04
Validation auc = 0.713615
Epoch 45
Validation auc = 0.659155
Epoch 46
Validation auc = 0.746479
Epoch 47
Validation auc = 0.691549
Epoch 48
Validation auc = 0.673239
Epoch 49
Loss = 8.5613e+00, PNorm = 70.3662, GNorm = 252.2920, lr_0 = 9.3122e-04
Validation auc = 0.740845
Epoch 50
Validation auc = 0.748122
Epoch 51
Validation auc = 0.714789
Epoch 52
Validation auc = 0.716197
Epoch 53
Validation auc = 0.684507
Epoch 54
Loss = 8.4205e+00, PNorm = 70.6389, GNorm = 730.0336, lr_0 = 9.2326e-04
Validation auc = 0.638263
Epoch 55
Validation auc = 0.717840
Epoch 56
Validation auc = 0.713850
Epoch 57
Validation auc = 0.620423
Epoch 58
Validation auc = 0.553991
Epoch 59
Loss = 2.1683e+01, PNorm = 71.1010, GNorm = 642.2538, lr_0 = 9.1537e-04
Validation auc = 0.697653
Epoch 60
Validation auc = 0.737089
Epoch 61
Validation auc = 0.738498
Epoch 62
Validation auc = 0.722535
Epoch 63
Validation auc = 0.722535
Epoch 64
Loss = 6.0752e+00, PNorm = 71.4978, GNorm = 628.1906, lr_0 = 9.0754e-04
Validation auc = 0.731690
Epoch 65
Validation auc = 0.709859
Epoch 66
Validation auc = 0.716901
Epoch 67
Validation auc = 0.725352
Epoch 68
Validation auc = 0.718075
Epoch 69
Loss = 6.4353e+00, PNorm = 71.7073, GNorm = 1398.7750, lr_0 = 8.9978e-04
Validation auc = 0.713380
Epoch 70
Validation auc = 0.693427
Epoch 71
Validation auc = 0.689671
Epoch 72
Validation auc = 0.658216
Epoch 73
Validation auc = 0.786150
Epoch 74
Loss = 5.7079e+00, PNorm = 71.9871, GNorm = 93.0965, lr_0 = 8.9209e-04
Validation auc = 0.765023
Epoch 75
Validation auc = 0.684038
Epoch 76
Validation auc = 0.744601
Epoch 77
Validation auc = 0.700469
Epoch 78
Validation auc = 0.674883
Epoch 79
Loss = 5.3465e+00, PNorm = 72.2895, GNorm = 420.6771, lr_0 = 8.8447e-04
Validation auc = 0.701408
Epoch 80
Validation auc = 0.625822
Epoch 81
Validation auc = 0.727700
Epoch 82
Validation auc = 0.720423
Epoch 83
Validation auc = 0.681690
Epoch 84
Loss = 7.7783e+00, PNorm = 72.6770, GNorm = 516.8802, lr_0 = 8.7691e-04
Validation auc = 0.734272
Epoch 85
Validation auc = 0.732160
Epoch 86
Validation auc = 0.704695
Epoch 87
Validation auc = 0.683803
Epoch 88
Validation auc = 0.692723
Epoch 89
Loss = 5.8368e+00, PNorm = 72.9630, GNorm = 675.6296, lr_0 = 8.6941e-04
Validation auc = 0.666197
Epoch 90
Validation auc = 0.654460
Epoch 91
Validation auc = 0.665728
Epoch 92
Validation auc = 0.639906
Epoch 93
Validation auc = 0.688967
Epoch 94
Loss = 4.8555e+00, PNorm = 73.1762, GNorm = 1036.3682, lr_0 = 8.6198e-04
Validation auc = 0.657277
Epoch 95
Validation auc = 0.666667
Epoch 96
Validation auc = 0.702347
Epoch 97
Validation auc = 0.675117
Epoch 98
Validation auc = 0.709155
Epoch 99
Loss = 6.7481e+00, PNorm = 73.4703, GNorm = 1287.2284, lr_0 = 8.5461e-04
Validation auc = 0.735446
Epoch 100
Validation auc = 0.724413
Epoch 101
Validation auc = 0.733803
Epoch 102
Validation auc = 0.697887
Epoch 103
Validation auc = 0.752347
Epoch 104
Loss = 2.9676e+00, PNorm = 73.7157, GNorm = 144.8280, lr_0 = 8.4730e-04
Validation auc = 0.736385
Epoch 105
Validation auc = 0.691315
Epoch 106
Validation auc = 0.760329
Epoch 107
Validation auc = 0.748592
Epoch 108
Validation auc = 0.715493
Epoch 109
Loss = 6.9075e+00, PNorm = 73.9981, GNorm = 820.1651, lr_0 = 8.4006e-04
Validation auc = 0.719249
Epoch 110
Validation auc = 0.727934
Epoch 111
Validation auc = 0.764554
Epoch 112
Validation auc = 0.754225
Epoch 113
Validation auc = 0.712207
Epoch 114
Loss = 5.0003e+00, PNorm = 74.2187, GNorm = 188.2524, lr_0 = 8.3288e-04
Validation auc = 0.716901
Epoch 115
Validation auc = 0.702817
Epoch 116
Validation auc = 0.720423
Epoch 117
Validation auc = 0.739437
Epoch 118
Validation auc = 0.688028
Epoch 119
Loss = 9.6040e+00, PNorm = 74.5430, GNorm = 97.7169, lr_0 = 8.2576e-04
Validation auc = 0.744366
Epoch 120
Validation auc = 0.701643
Epoch 121
Validation auc = 0.680282
Epoch 122
Validation auc = 0.723474
Epoch 123
Validation auc = 0.731221
Epoch 124
Loss = 6.7675e+00, PNorm = 74.8617, GNorm = 285.5493, lr_0 = 8.1870e-04
Validation auc = 0.738498
Epoch 125
Validation auc = 0.731455
Epoch 126
Validation auc = 0.737089
Epoch 127
Validation auc = 0.733333
Epoch 128
Validation auc = 0.685211
Epoch 129
Loss = 5.1385e+00, PNorm = 75.0291, GNorm = 774.4083, lr_0 = 8.1170e-04
Validation auc = 0.749531
Epoch 130
Validation auc = 0.754460
Epoch 131
Validation auc = 0.667371
Epoch 132
Validation auc = 0.709624
Epoch 133
Validation auc = 0.688967
Epoch 134
Loss = 1.0948e+01, PNorm = 75.4735, GNorm = 2222.9678, lr_0 = 8.0476e-04
Validation auc = 0.725352
Epoch 135
Validation auc = 0.707981
Epoch 136
Validation auc = 0.696948
Epoch 137
Validation auc = 0.737089
Epoch 138
Validation auc = 0.733099
Epoch 139
Loss = 3.9086e+00, PNorm = 75.9090, GNorm = 219.6131, lr_0 = 7.9788e-04
Validation auc = 0.719484
Epoch 140
Validation auc = 0.727934
Epoch 141
Validation auc = 0.734742
Epoch 142
Validation auc = 0.737793
Epoch 143
Validation auc = 0.759624
Epoch 144
Loss = 4.3769e+00, PNorm = 76.2119, GNorm = 912.1712, lr_0 = 7.9106e-04
Validation auc = 0.753286
Epoch 145
Validation auc = 0.773239
Epoch 146
Validation auc = 0.710329
Epoch 147
Validation auc = 0.741080
Epoch 148
Validation auc = 0.749296
Epoch 149
Loss = 4.5202e+00, PNorm = 76.4542, GNorm = 208.1953, lr_0 = 7.8430e-04
Validation auc = 0.704225
Epoch 150
Validation auc = 0.749296
Epoch 151
Validation auc = 0.717371
Epoch 152
Validation auc = 0.714319
Epoch 153
Validation auc = 0.746244
Epoch 154
Loss = 2.6593e+00, PNorm = 76.7635, GNorm = 116.2151, lr_0 = 7.7759e-04
Validation auc = 0.677700
Epoch 155
Validation auc = 0.711737
Epoch 156
Validation auc = 0.689671
Epoch 157
Validation auc = 0.704930
Epoch 158
Validation auc = 0.730282
Epoch 159
Loss = 5.0362e+00, PNorm = 77.0928, GNorm = 363.0923, lr_0 = 7.7095e-04
Validation auc = 0.718075
Epoch 160
Validation auc = 0.714085
Epoch 161
Validation auc = 0.695775
Epoch 162
Validation auc = 0.694131
Epoch 163
Validation auc = 0.722066
Epoch 164
Loss = 1.9752e+00, PNorm = 77.4132, GNorm = 276.3800, lr_0 = 7.6436e-04
Validation auc = 0.677230
Epoch 165
Validation auc = 0.720188
Epoch 166
Validation auc = 0.714085
Epoch 167
Validation auc = 0.665023
Epoch 168
Validation auc = 0.714319
Epoch 169
Loss = 3.9505e+00, PNorm = 77.6630, GNorm = 393.5066, lr_0 = 7.5782e-04
Validation auc = 0.675117
Epoch 170
Validation auc = 0.680986
Epoch 171
Validation auc = 0.725822
Epoch 172
Validation auc = 0.728169
Epoch 173
Validation auc = 0.676526
Epoch 174
Loss = 1.0179e+01, PNorm = 78.0498, GNorm = 1324.6846, lr_0 = 7.5134e-04
Validation auc = 0.723944
Epoch 175
Validation auc = 0.674178
Epoch 176
Validation auc = 0.658451
Epoch 177
Validation auc = 0.735211
Epoch 178
Validation auc = 0.738028
Epoch 179
Loss = 2.0367e+00, PNorm = 78.5544, GNorm = 376.6123, lr_0 = 7.4492e-04
Validation auc = 0.662207
Epoch 180
Validation auc = 0.694836
Epoch 181
Validation auc = 0.718779
Epoch 182
Validation auc = 0.674413
Epoch 183
Validation auc = 0.694131
Epoch 184
Loss = 3.1833e+00, PNorm = 79.0136, GNorm = 932.8360, lr_0 = 7.3855e-04
Validation auc = 0.680282
Epoch 185
Validation auc = 0.664554
Epoch 186
Validation auc = 0.704695
Epoch 187
Validation auc = 0.700235
Epoch 188
Validation auc = 0.655634
Epoch 189
Loss = 1.3972e+00, PNorm = 79.4038, GNorm = 35.0877, lr_0 = 7.3224e-04
Validation auc = 0.715962
Epoch 190
Validation auc = 0.639671
Epoch 191
Validation auc = 0.738732
Epoch 192
Validation auc = 0.726761
Epoch 193
Validation auc = 0.660094
Epoch 194
Loss = 1.8226e+00, PNorm = 79.7337, GNorm = 321.4355, lr_0 = 7.2598e-04
Validation auc = 0.733803
Epoch 195
Validation auc = 0.751643
Epoch 196
Validation auc = 0.713850
Epoch 197
Validation auc = 0.673239
Epoch 198
Validation auc = 0.746009
Epoch 199
Loss = 7.4424e+00, PNorm = 79.9189, GNorm = 808.5800, lr_0 = 7.1977e-04
Validation auc = 0.739437
Epoch 200
Validation auc = 0.679343
Epoch 201
Validation auc = 0.757042
Epoch 202
Validation auc = 0.725822
Epoch 203
Validation auc = 0.709624
Epoch 204
Loss = 7.1915e-01, PNorm = 80.2510, GNorm = 64.1892, lr_0 = 7.1362e-04
Validation auc = 0.745070
Epoch 205
Validation auc = 0.732160
Epoch 206
Validation auc = 0.708920
Epoch 207
Validation auc = 0.709859
Epoch 208
Validation auc = 0.721831
Epoch 209
Loss = 2.4794e+00, PNorm = 80.5952, GNorm = 93.3624, lr_0 = 7.0752e-04
Validation auc = 0.710329
Epoch 210
Validation auc = 0.692019
Epoch 211
Validation auc = 0.707277
Epoch 212
Validation auc = 0.692019
Epoch 213
Validation auc = 0.673709
Epoch 214
Loss = 1.4936e+00, PNorm = 80.8731, GNorm = 328.6869, lr_0 = 7.0147e-04
Validation auc = 0.694601
Epoch 215
Validation auc = 0.699765
Epoch 216
Validation auc = 0.723944
Epoch 217
Validation auc = 0.658216
Epoch 218
Validation auc = 0.705399
Epoch 219
Loss = 2.9693e+00, PNorm = 81.1497, GNorm = 673.6511, lr_0 = 6.9548e-04
Validation auc = 0.734272
Epoch 220
Validation auc = 0.669249
Epoch 221
Validation auc = 0.709390
Epoch 222
Validation auc = 0.725587
Epoch 223
Validation auc = 0.665728
Epoch 224
Loss = 2.9425e+00, PNorm = 81.5385, GNorm = 880.7263, lr_0 = 6.8953e-04
Validation auc = 0.692019
Epoch 225
Validation auc = 0.721596
Epoch 226
Validation auc = 0.676995
Epoch 227
Validation auc = 0.625587
Epoch 228
Validation auc = 0.677230
Epoch 229
Loss = 5.4851e+00, PNorm = 82.0735, GNorm = 424.7801, lr_0 = 6.8364e-04
Validation auc = 0.684742
Epoch 230
Validation auc = 0.673944
Epoch 231
Validation auc = 0.670423
Epoch 232
Validation auc = 0.664789
Epoch 233
Validation auc = 0.702582
Epoch 234
Loss = 2.3684e+00, PNorm = 82.5114, GNorm = 377.6558, lr_0 = 6.7779e-04
Validation auc = 0.711737
Epoch 235
Validation auc = 0.639906
Epoch 236
Validation auc = 0.666197
Epoch 237
Validation auc = 0.710798
Epoch 238
Validation auc = 0.715023
Epoch 239
Loss = 3.6903e+00, PNorm = 82.9695, GNorm = 117.2030, lr_0 = 6.7200e-04
Validation auc = 0.703756
Epoch 240
Validation auc = 0.690610
Epoch 241
Validation auc = 0.734272
Epoch 242
Validation auc = 0.729812
Epoch 243
Validation auc = 0.676995
Epoch 244
Loss = 1.2454e+00, PNorm = 83.3601, GNorm = 378.2423, lr_0 = 6.6625e-04
Validation auc = 0.660563
Epoch 245
Validation auc = 0.715728
Epoch 246
Validation auc = 0.722066
Epoch 247
Validation auc = 0.681221
Epoch 248
Validation auc = 0.648357
Epoch 249
Loss = 8.2199e-01, PNorm = 83.6534, GNorm = 33.2586, lr_0 = 6.6056e-04
Validation auc = 0.721127
Epoch 250
Validation auc = 0.672300
Epoch 251
Validation auc = 0.628169
Epoch 252
Validation auc = 0.690141
Epoch 253
Validation auc = 0.666667
Epoch 254
Loss = 4.6170e+00, PNorm = 83.9868, GNorm = 295.5171, lr_0 = 6.5491e-04
Validation auc = 0.676761
Epoch 255
Validation auc = 0.611737
Epoch 256
Validation auc = 0.670892
Epoch 257
Validation auc = 0.696948
Epoch 258
Validation auc = 0.676056
Epoch 259
Loss = 8.5108e-01, PNorm = 84.4016, GNorm = 439.5606, lr_0 = 6.4931e-04
Validation auc = 0.627700
Epoch 260
Validation auc = 0.656338
Epoch 261
Validation auc = 0.665962
Epoch 262
Validation auc = 0.634272
Epoch 263
Validation auc = 0.626056
Epoch 264
Loss = 1.9282e+00, PNorm = 84.8450, GNorm = 296.9421, lr_0 = 6.4376e-04
Validation auc = 0.617840
Epoch 265
Validation auc = 0.636620
Epoch 266
Validation auc = 0.642958
Epoch 267
Validation auc = 0.649765
Epoch 268
Validation auc = 0.650235
Epoch 269
Loss = 8.6426e-01, PNorm = 85.1788, GNorm = 103.0397, lr_0 = 6.3826e-04
Validation auc = 0.651643
Epoch 270
Validation auc = 0.642254
Epoch 271
Validation auc = 0.639906
Epoch 272
Validation auc = 0.634038
Epoch 273
Validation auc = 0.634038
Epoch 274
Loss = 1.4983e+00, PNorm = 85.4705, GNorm = 365.5731, lr_0 = 6.3280e-04
Validation auc = 0.640845
Epoch 275
Validation auc = 0.637089
Epoch 276
Validation auc = 0.655399
Epoch 277
Validation auc = 0.655164
Epoch 278
Validation auc = 0.647418
Epoch 279
Loss = 1.2353e+00, PNorm = 85.7628, GNorm = 489.8201, lr_0 = 6.2739e-04
Validation auc = 0.644601
Epoch 280
Validation auc = 0.657042
Epoch 281
Validation auc = 0.628873
Epoch 282
Validation auc = 0.629577
Epoch 283
Validation auc = 0.634742
Epoch 284
Loss = 4.6337e-01, PNorm = 86.0354, GNorm = 127.1430, lr_0 = 6.2203e-04
Validation auc = 0.636854
Epoch 285
Validation auc = 0.631925
Epoch 286
Validation auc = 0.656338
Epoch 287
Validation auc = 0.653756
Epoch 288
Validation auc = 0.656338
Epoch 289
Loss = 1.1826e+00, PNorm = 86.2856, GNorm = 117.0814, lr_0 = 6.1671e-04
Validation auc = 0.630516
Epoch 290
Validation auc = 0.659155
Epoch 291
Validation auc = 0.667371
Epoch 292
Validation auc = 0.649765
Epoch 293
Validation auc = 0.620657
Epoch 294
Loss = 1.6674e+00, PNorm = 86.5882, GNorm = 0.0002, lr_0 = 6.1144e-04
Validation auc = 0.692254
Epoch 295
Validation auc = 0.664789
Epoch 296
Validation auc = 0.650704
Epoch 297
Validation auc = 0.622535
Epoch 298
Validation auc = 0.637559
Epoch 299
Loss = 1.9637e+00, PNorm = 86.8358, GNorm = 133.8680, lr_0 = 6.0621e-04
Validation auc = 0.693662
Model 0 best validation auc = 0.791080 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.752559
Ensemble test auc = 0.752559
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.494836
Epoch 1
Validation auc = 0.688732
Epoch 2
Validation auc = 0.744366
Epoch 3
Validation auc = 0.742723
Epoch 4
Loss = 2.2425e+01, PNorm = 68.3663, GNorm = 2017.9367, lr_0 = 6.5000e-04
Validation auc = 0.738028
Epoch 5
Validation auc = 0.640376
Epoch 6
Validation auc = 0.724648
Epoch 7
Validation auc = 0.725587
Epoch 8
Validation auc = 0.670892
Epoch 9
Loss = 1.9797e+01, PNorm = 68.5110, GNorm = 1377.5302, lr_0 = 9.9743e-04
Validation auc = 0.727230
Epoch 10
Validation auc = 0.628169
Epoch 11
Validation auc = 0.676056
Epoch 12
Validation auc = 0.646948
Epoch 13
Validation auc = 0.627230
Epoch 14
Loss = 1.3526e+01, PNorm = 68.6807, GNorm = 1321.7147, lr_0 = 9.8890e-04
Validation auc = 0.531690
Epoch 15
Validation auc = 0.613615
Epoch 16
Validation auc = 0.582864
Epoch 17
Validation auc = 0.673709
Epoch 18
Validation auc = 0.634742
Epoch 19
Loss = 3.0150e+01, PNorm = 68.8194, GNorm = 197.9541, lr_0 = 9.8045e-04
Validation auc = 0.541315
Epoch 20
Validation auc = 0.687793
Epoch 21
Validation auc = 0.701408
Epoch 22
Validation auc = 0.530282
Epoch 23
Validation auc = 0.606573
Epoch 24
Loss = 2.9150e+01, PNorm = 68.9498, GNorm = 2049.2871, lr_0 = 9.7207e-04
Validation auc = 0.679577
Epoch 25
Validation auc = 0.558451
Epoch 26
Validation auc = 0.680282
Epoch 27
Validation auc = 0.622066
Epoch 28
Validation auc = 0.543662
Epoch 29
Loss = 2.8424e+01, PNorm = 69.0613, GNorm = 2507.6670, lr_0 = 9.6376e-04
Validation auc = 0.648357
Epoch 30
Validation auc = 0.634272
Epoch 31
Validation auc = 0.696948
Epoch 32
Validation auc = 0.557746
Epoch 33
Validation auc = 0.676526
Epoch 34
Loss = 3.0829e+01, PNorm = 69.2246, GNorm = 2738.5029, lr_0 = 9.5552e-04
Validation auc = 0.737324
Epoch 35
Validation auc = 0.592958
Epoch 36
Validation auc = 0.726526
Epoch 37
Validation auc = 0.657746
Epoch 38
Validation auc = 0.620892
Epoch 39
Loss = 2.0116e+01, PNorm = 69.4872, GNorm = 1399.8276, lr_0 = 9.4735e-04
Validation auc = 0.598592
Epoch 40
Validation auc = 0.651174
Epoch 41
Validation auc = 0.654695
Epoch 42
Validation auc = 0.593192
Epoch 43
Validation auc = 0.721362
Epoch 44
Loss = 7.2489e+00, PNorm = 69.7258, GNorm = 730.1607, lr_0 = 9.3925e-04
Validation auc = 0.659624
Epoch 45
Validation auc = 0.718310
Epoch 46
Validation auc = 0.691315
Epoch 47
Validation auc = 0.706573
Epoch 48
Validation auc = 0.692019
Epoch 49
Loss = 8.4171e+00, PNorm = 70.1042, GNorm = 239.9574, lr_0 = 9.3122e-04
Validation auc = 0.698826
Epoch 50
Validation auc = 0.697653
Epoch 51
Validation auc = 0.651408
Epoch 52
Validation auc = 0.636150
Epoch 53
Validation auc = 0.656103
Epoch 54
Loss = 6.2689e+00, PNorm = 70.4506, GNorm = 397.7169, lr_0 = 9.2326e-04
Validation auc = 0.639906
Epoch 55
Validation auc = 0.710094
Epoch 56
Validation auc = 0.684507
Epoch 57
Validation auc = 0.615962
Epoch 58
Validation auc = 0.717840
Epoch 59
Loss = 9.3669e+00, PNorm = 70.8845, GNorm = 529.5323, lr_0 = 9.1537e-04
Validation auc = 0.714085
Epoch 60
Validation auc = 0.702113
Epoch 61
Validation auc = 0.691080
Epoch 62
Validation auc = 0.693427
Epoch 63
Validation auc = 0.710798
Epoch 64
Loss = 4.8675e+00, PNorm = 71.2207, GNorm = 261.1501, lr_0 = 9.0754e-04
Validation auc = 0.725587
Epoch 65
Validation auc = 0.721127
Epoch 66
Validation auc = 0.648826
Epoch 67
Validation auc = 0.724883
Epoch 68
Validation auc = 0.710798
Epoch 69
Loss = 6.2846e+00, PNorm = 71.5156, GNorm = 887.4053, lr_0 = 8.9978e-04
Validation auc = 0.701643
Epoch 70
Validation auc = 0.714554
Epoch 71
Validation auc = 0.692019
Epoch 72
Validation auc = 0.705164
Epoch 73
Validation auc = 0.667371
Epoch 74
Loss = 9.7365e+00, PNorm = 71.6968, GNorm = 917.1886, lr_0 = 8.9209e-04
Validation auc = 0.673474
Epoch 75
Validation auc = 0.727700
Epoch 76
Validation auc = 0.537089
Epoch 77
Validation auc = 0.689906
Epoch 78
Validation auc = 0.690845
Epoch 79
Loss = 7.8108e+00, PNorm = 71.9710, GNorm = 389.3274, lr_0 = 8.8447e-04
Validation auc = 0.691080
Epoch 80
Validation auc = 0.758685
Epoch 81
Validation auc = 0.723944
Epoch 82
Validation auc = 0.757512
Epoch 83
Validation auc = 0.718545
Epoch 84
Loss = 5.5438e+00, PNorm = 72.1413, GNorm = 201.7457, lr_0 = 8.7691e-04
Validation auc = 0.703052
Epoch 85
Validation auc = 0.657042
Epoch 86
Validation auc = 0.731455
Epoch 87
Validation auc = 0.716901
Epoch 88
Validation auc = 0.628638
Epoch 89
Loss = 1.5715e+01, PNorm = 72.2357, GNorm = 2150.3163, lr_0 = 8.6941e-04
Validation auc = 0.711268
Epoch 90
Validation auc = 0.635681
Epoch 91
Validation auc = 0.659624
Epoch 92
Validation auc = 0.699061
Epoch 93
Validation auc = 0.771596
Epoch 94
Loss = 9.0201e+00, PNorm = 72.4867, GNorm = 1003.8044, lr_0 = 8.6198e-04
Validation auc = 0.723944
Epoch 95
Validation auc = 0.697418
Epoch 96
Validation auc = 0.772066
Epoch 97
Validation auc = 0.665962
Epoch 98
Validation auc = 0.765728
Epoch 99
Loss = 3.4766e+00, PNorm = 72.6114, GNorm = 594.9681, lr_0 = 8.5461e-04
Validation auc = 0.682629
Epoch 100
Validation auc = 0.755869
Epoch 101
Validation auc = 0.751408
Epoch 102
Validation auc = 0.727230
Epoch 103
Validation auc = 0.710329
Epoch 104
Loss = 3.1288e+00, PNorm = 72.8315, GNorm = 134.4000, lr_0 = 8.4730e-04
Validation auc = 0.705164
Epoch 105
Validation auc = 0.723709
Epoch 106
Validation auc = 0.700704
Epoch 107
Validation auc = 0.702347
Epoch 108
Validation auc = 0.705634
Epoch 109
Loss = 7.6881e+00, PNorm = 73.0133, GNorm = 825.9210, lr_0 = 8.4006e-04
Validation auc = 0.719953
Epoch 110
Validation auc = 0.596009
Epoch 111
Validation auc = 0.704460
Epoch 112
Validation auc = 0.735681
Epoch 113
Validation auc = 0.730751
Epoch 114
Loss = 3.7746e+00, PNorm = 73.0726, GNorm = 313.1537, lr_0 = 8.3288e-04
Validation auc = 0.732864
Epoch 115
Validation auc = 0.756103
Epoch 116
Validation auc = 0.758920
Epoch 117
Validation auc = 0.668075
Epoch 118
Validation auc = 0.762441
Epoch 119
Loss = 6.5139e+00, PNorm = 73.1982, GNorm = 1225.2573, lr_0 = 8.2576e-04
Validation auc = 0.744366
Epoch 120
Validation auc = 0.666432
Epoch 121
Validation auc = 0.711502
Epoch 122
Validation auc = 0.707981
Epoch 123
Validation auc = 0.673709
Epoch 124
Loss = 5.0671e+00, PNorm = 73.3835, GNorm = 695.1045, lr_0 = 8.1870e-04
Validation auc = 0.717840
Epoch 125
Validation auc = 0.689671
Epoch 126
Validation auc = 0.670892
Epoch 127
Validation auc = 0.702817
Epoch 128
Validation auc = 0.791315
Epoch 129
Loss = 2.2180e+00, PNorm = 73.5203, GNorm = 408.4899, lr_0 = 8.1170e-04
Validation auc = 0.757277
Epoch 130
Validation auc = 0.722770
Epoch 131
Validation auc = 0.769014
Epoch 132
Validation auc = 0.759624
Epoch 133
Validation auc = 0.735681
Epoch 134
Loss = 3.0896e+00, PNorm = 73.6091, GNorm = 800.3952, lr_0 = 8.0476e-04
Validation auc = 0.744131
Epoch 135
Validation auc = 0.741315
Epoch 136
Validation auc = 0.668779
Epoch 137
Validation auc = 0.697887
Epoch 138
Validation auc = 0.736620
Epoch 139
Loss = 1.8405e+00, PNorm = 73.7241, GNorm = 128.8558, lr_0 = 7.9788e-04
Validation auc = 0.663380
Epoch 140
Validation auc = 0.658920
Epoch 141
Validation auc = 0.672770
Epoch 142
Validation auc = 0.678404
Epoch 143
Validation auc = 0.688732
Epoch 144
Loss = 2.3723e+00, PNorm = 73.9375, GNorm = 231.3507, lr_0 = 7.9106e-04
Validation auc = 0.728169
Epoch 145
Validation auc = 0.671831
Epoch 146
Validation auc = 0.778638
Epoch 147
Validation auc = 0.708451
Epoch 148
Validation auc = 0.678404
Epoch 149
Loss = 2.5642e+00, PNorm = 74.0730, GNorm = 160.2951, lr_0 = 7.8430e-04
Validation auc = 0.734507
Epoch 150
Validation auc = 0.704225
Epoch 151
Validation auc = 0.764789
Epoch 152
Validation auc = 0.670423
Epoch 153
Validation auc = 0.677230
Epoch 154
Loss = 2.0826e+00, PNorm = 74.2373, GNorm = 136.7328, lr_0 = 7.7759e-04
Validation auc = 0.721596
Epoch 155
Validation auc = 0.718075
Epoch 156
Validation auc = 0.676291
Epoch 157
Validation auc = 0.760094
Epoch 158
Validation auc = 0.722535
Epoch 159
Loss = 3.8676e+00, PNorm = 74.4943, GNorm = 385.3788, lr_0 = 7.7095e-04
Validation auc = 0.676761
Epoch 160
Validation auc = 0.722066
Epoch 161
Validation auc = 0.702347
Epoch 162
Validation auc = 0.682160
Epoch 163
Validation auc = 0.730516
Epoch 164
Loss = 3.9224e+00, PNorm = 74.6518, GNorm = 672.6973, lr_0 = 7.6436e-04
Validation auc = 0.722770
Epoch 165
Validation auc = 0.695775
Epoch 166
Validation auc = 0.750469
Epoch 167
Validation auc = 0.764085
Epoch 168
Validation auc = 0.689202
Epoch 169
Loss = 7.8524e+00, PNorm = 74.8070, GNorm = 1108.7698, lr_0 = 7.5782e-04
Validation auc = 0.779812
Epoch 170
Validation auc = 0.778404
Epoch 171
Validation auc = 0.686150
Epoch 172
Validation auc = 0.780986
Epoch 173
Validation auc = 0.757981
Epoch 174
Loss = 3.1311e+00, PNorm = 75.0542, GNorm = 269.7254, lr_0 = 7.5134e-04
Validation auc = 0.710798
Epoch 175
Validation auc = 0.732160
Epoch 176
Validation auc = 0.760563
Epoch 177
Validation auc = 0.722066
Epoch 178
Validation auc = 0.683099
Epoch 179
Loss = 5.0908e+00, PNorm = 75.3512, GNorm = 128.1026, lr_0 = 7.4492e-04
Validation auc = 0.743662
Epoch 180
Validation auc = 0.731455
Epoch 181
Validation auc = 0.682394
Epoch 182
Validation auc = 0.710329
Epoch 183
Validation auc = 0.627934
Epoch 184
Loss = 1.2095e+01, PNorm = 75.6606, GNorm = 751.5549, lr_0 = 7.3855e-04
Validation auc = 0.707512
Epoch 185
Validation auc = 0.667371
Epoch 186
Validation auc = 0.713146
Epoch 187
Validation auc = 0.691784
Epoch 188
Validation auc = 0.724413
Epoch 189
Loss = 6.8199e+00, PNorm = 76.0838, GNorm = 325.2994, lr_0 = 7.3224e-04
Validation auc = 0.673005
Epoch 190
Validation auc = 0.693897
Epoch 191
Validation auc = 0.701643
Epoch 192
Validation auc = 0.725117
Epoch 193
Validation auc = 0.673239
Epoch 194
Loss = 7.4034e+00, PNorm = 76.5184, GNorm = 421.2965, lr_0 = 7.2598e-04
Validation auc = 0.701408
Epoch 195
Validation auc = 0.742254
Epoch 196
Validation auc = 0.744836
Epoch 197
Validation auc = 0.717840
Epoch 198
Validation auc = 0.712676
Epoch 199
Loss = 4.6396e+00, PNorm = 76.8549, GNorm = 248.5300, lr_0 = 7.1977e-04
Validation auc = 0.759624
Epoch 200
Validation auc = 0.762911
Epoch 201
Validation auc = 0.731221
Epoch 202
Validation auc = 0.715728
Epoch 203
Validation auc = 0.755869
Epoch 204
Loss = 3.1775e+00, PNorm = 77.1587, GNorm = 337.0469, lr_0 = 7.1362e-04
Validation auc = 0.750704
Epoch 205
Validation auc = 0.725822
Epoch 206
Validation auc = 0.690610
Epoch 207
Validation auc = 0.747887
Epoch 208
Validation auc = 0.759155
Epoch 209
Loss = 1.6958e+00, PNorm = 77.4179, GNorm = 113.5304, lr_0 = 7.0752e-04
Validation auc = 0.711268
Epoch 210
Validation auc = 0.712207
Epoch 211
Validation auc = 0.746009
Epoch 212
Validation auc = 0.765962
Epoch 213
Validation auc = 0.737089
Epoch 214
Loss = 3.0206e+00, PNorm = 77.6351, GNorm = 83.9306, lr_0 = 7.0147e-04
Validation auc = 0.738263
Epoch 215
Validation auc = 0.728169
Epoch 216
Validation auc = 0.722066
Epoch 217
Validation auc = 0.759155
Epoch 218
Validation auc = 0.752113
Epoch 219
Loss = 8.9136e-01, PNorm = 77.7561, GNorm = 131.2018, lr_0 = 6.9548e-04
Validation auc = 0.740845
Epoch 220
Validation auc = 0.766432
Epoch 221
Validation auc = 0.760094
Epoch 222
Validation auc = 0.691549
Epoch 223
Validation auc = 0.713615
Epoch 224
Loss = 1.6626e+00, PNorm = 77.8819, GNorm = 60.8514, lr_0 = 6.8953e-04
Validation auc = 0.766197
Epoch 225
Validation auc = 0.704695
Epoch 226
Validation auc = 0.669014
Epoch 227
Validation auc = 0.775822
Epoch 228
Validation auc = 0.776526
Epoch 229
Loss = 1.2017e+00, PNorm = 78.0943, GNorm = 232.4539, lr_0 = 6.8364e-04
Validation auc = 0.686620
Epoch 230
Validation auc = 0.770423
Epoch 231
Validation auc = 0.780047
Epoch 232
Validation auc = 0.778638
Epoch 233
Validation auc = 0.707042
Epoch 234
Loss = 1.9662e+00, PNorm = 78.4203, GNorm = 570.3130, lr_0 = 6.7779e-04
Validation auc = 0.699531
Epoch 235
Validation auc = 0.783568
Epoch 236
Validation auc = 0.769014
Epoch 237
Validation auc = 0.739906
Epoch 238
Validation auc = 0.737559
Epoch 239
Loss = 1.9088e+00, PNorm = 78.6288, GNorm = 162.7602, lr_0 = 6.7200e-04
Validation auc = 0.764554
Epoch 240
Validation auc = 0.770657
Epoch 241
Validation auc = 0.721596
Epoch 242
Validation auc = 0.746948
Epoch 243
Validation auc = 0.759155
Epoch 244
Loss = 3.0694e+00, PNorm = 78.8044, GNorm = 21.7898, lr_0 = 6.6625e-04
Validation auc = 0.759155
Epoch 245
Validation auc = 0.759155
Epoch 246
Validation auc = 0.679577
Epoch 247
Validation auc = 0.707746
Epoch 248
Validation auc = 0.749765
Epoch 249
Loss = 5.4522e+00, PNorm = 79.0046, GNorm = 547.4526, lr_0 = 6.6056e-04
Validation auc = 0.778404
Epoch 250
Validation auc = 0.713850
Epoch 251
Validation auc = 0.698357
Epoch 252
Validation auc = 0.781690
Epoch 253
Validation auc = 0.790845
Epoch 254
Loss = 1.5540e+00, PNorm = 79.1850, GNorm = 78.2909, lr_0 = 6.5491e-04
Validation auc = 0.756573
Epoch 255
Validation auc = 0.730282
Epoch 256
Validation auc = 0.758216
Epoch 257
Validation auc = 0.786854
Epoch 258
Validation auc = 0.764319
Epoch 259
Loss = 1.0096e+00, PNorm = 79.3473, GNorm = 218.9011, lr_0 = 6.4931e-04
Validation auc = 0.770423
Epoch 260
Validation auc = 0.767606
Epoch 261
Validation auc = 0.775117
Epoch 262
Validation auc = 0.722066
Epoch 263
Validation auc = 0.748357
Epoch 264
Loss = 1.0227e+00, PNorm = 79.5227, GNorm = 174.4503, lr_0 = 6.4376e-04
Validation auc = 0.755869
Epoch 265
Validation auc = 0.688967
Epoch 266
Validation auc = 0.704225
Epoch 267
Validation auc = 0.723709
Epoch 268
Validation auc = 0.723474
Epoch 269
Loss = 1.2535e+00, PNorm = 79.6819, GNorm = 124.0220, lr_0 = 6.3826e-04
Validation auc = 0.722066
Epoch 270
Validation auc = 0.711972
Epoch 271
Validation auc = 0.742019
Epoch 272
Validation auc = 0.727230
Epoch 273
Validation auc = 0.782394
Epoch 274
Loss = 1.6725e+00, PNorm = 79.8712, GNorm = 330.5289, lr_0 = 6.3280e-04
Validation auc = 0.746714
Epoch 275
Validation auc = 0.712911
Epoch 276
Validation auc = 0.714319
Epoch 277
Validation auc = 0.736150
Epoch 278
Validation auc = 0.734742
Epoch 279
Loss = 1.5412e+00, PNorm = 80.0098, GNorm = 163.2851, lr_0 = 6.2739e-04
Validation auc = 0.741549
Epoch 280
Validation auc = 0.730516
Epoch 281
Validation auc = 0.689202
Epoch 282
Validation auc = 0.735446
Epoch 283
Validation auc = 0.749765
Epoch 284
Loss = 3.9074e+00, PNorm = 80.2544, GNorm = 397.4370, lr_0 = 6.2203e-04
Validation auc = 0.738028
Epoch 285
Validation auc = 0.680986
Epoch 286
Validation auc = 0.761502
Epoch 287
Validation auc = 0.757277
Epoch 288
Validation auc = 0.734742
Epoch 289
Loss = 6.4334e-01, PNorm = 80.5157, GNorm = 87.2062, lr_0 = 6.1671e-04
Validation auc = 0.721596
Epoch 290
Validation auc = 0.699765
Epoch 291
Validation auc = 0.736620
Epoch 292
Validation auc = 0.737089
Epoch 293
Validation auc = 0.683803
Epoch 294
Loss = 4.4081e+00, PNorm = 80.7643, GNorm = 876.5839, lr_0 = 6.1144e-04
Validation auc = 0.688498
Epoch 295
Validation auc = 0.706808
Epoch 296
Validation auc = 0.684977
Epoch 297
Validation auc = 0.730047
Epoch 298
Validation auc = 0.641784
Epoch 299
Loss = 5.9292e+00, PNorm = 81.1651, GNorm = 694.9058, lr_0 = 6.0621e-04
Validation auc = 0.691315
Model 0 best validation auc = 0.791315 on epoch 128
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.812161
Ensemble test auc = 0.812161
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1400, bias=False)
        (W_h): Linear(in_features=1400, out_features=1400, bias=False)
        (W_o): Linear(in_features=1533, out_features=1400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=1700, out_features=1400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1400, out_features=1, bias=True)
  )
)
Number of parameters = 6,696,201
Moving model to cuda
Epoch 0
Validation auc = 0.652582
Epoch 1
Validation auc = 0.498122
Epoch 2
Validation auc = 0.587793
Epoch 3
Validation auc = 0.776995
Epoch 4
Loss = 2.3152e+01, PNorm = 68.3678, GNorm = 1970.2494, lr_0 = 6.5000e-04
Validation auc = 0.570892
Epoch 5
Validation auc = 0.786620
Epoch 6
Validation auc = 0.745540
Epoch 7
Validation auc = 0.546948
Epoch 8
Validation auc = 0.703756
Epoch 9
Loss = 2.1442e+01, PNorm = 68.5373, GNorm = 2177.1328, lr_0 = 9.9743e-04
Validation auc = 0.757512
Epoch 10
Validation auc = 0.648826
Epoch 11
Validation auc = 0.691549
Epoch 12
Validation auc = 0.684272
Epoch 13
Validation auc = 0.688732
Epoch 14
Loss = 2.1733e+01, PNorm = 68.7134, GNorm = 2357.2786, lr_0 = 9.8890e-04
Validation auc = 0.582394
Epoch 15
Validation auc = 0.606808
Epoch 16
Validation auc = 0.724883
Epoch 17
Validation auc = 0.580282
Epoch 18
Validation auc = 0.657277
Epoch 19
Loss = 1.7531e+01, PNorm = 68.8337, GNorm = 742.2331, lr_0 = 9.8045e-04
Validation auc = 0.706103
Epoch 20
Validation auc = 0.664789
Epoch 21
Validation auc = 0.677934
Epoch 22
Validation auc = 0.683333
Epoch 23
Validation auc = 0.704930
Epoch 24
Loss = 9.4289e+00, PNorm = 68.9313, GNorm = 1415.2708, lr_0 = 9.7207e-04
Validation auc = 0.713146
Epoch 25
Validation auc = 0.592488
Epoch 26
Validation auc = 0.647887
Epoch 27
Validation auc = 0.667371
Epoch 28
Validation auc = 0.656103
Epoch 29
Loss = 1.3586e+01, PNorm = 69.0327, GNorm = 361.8346, lr_0 = 9.6376e-04
Validation auc = 0.636150
Epoch 30
Validation auc = 0.630047
Epoch 31
Validation auc = 0.671362
Epoch 32
Validation auc = 0.716432
Epoch 33
Validation auc = 0.668779
Epoch 34
Loss = 1.5972e+01, PNorm = 69.1630, GNorm = 765.1678, lr_0 = 9.5552e-04
Validation auc = 0.709859
Epoch 35
Validation auc = 0.701174
Epoch 36
Validation auc = 0.718779
Epoch 37
Validation auc = 0.727465
Epoch 38
Validation auc = 0.711737
Epoch 39
Loss = 7.0255e+00, PNorm = 69.3434, GNorm = 131.0743, lr_0 = 9.4735e-04
Validation auc = 0.614319
Epoch 40
Validation auc = 0.646009
Epoch 41
Validation auc = 0.672300
Epoch 42
Validation auc = 0.684507
Epoch 43
Validation auc = 0.665258
Epoch 44
Loss = 7.9585e+00, PNorm = 69.5222, GNorm = 834.7091, lr_0 = 9.3925e-04
Validation auc = 0.625352
Epoch 45
Validation auc = 0.674648
Epoch 46
Validation auc = 0.676056
Epoch 47
Validation auc = 0.684038
Epoch 48
Validation auc = 0.672300
Epoch 49
Loss = 8.2754e+00, PNorm = 69.6555, GNorm = 423.2699, lr_0 = 9.3122e-04
Validation auc = 0.620188
Epoch 50
Validation auc = 0.636150
Epoch 51
Validation auc = 0.624648
Epoch 52
Validation auc = 0.618779
Epoch 53
Validation auc = 0.659155
Epoch 54
Loss = 9.9203e+00, PNorm = 69.8205, GNorm = 1041.3381, lr_0 = 9.2326e-04
Validation auc = 0.699531
Epoch 55
Validation auc = 0.684507
Epoch 56
Validation auc = 0.705164
Epoch 57
Validation auc = 0.661033
Epoch 58
Validation auc = 0.608216
Epoch 59
Loss = 1.2240e+01, PNorm = 69.9803, GNorm = 1400.7695, lr_0 = 9.1537e-04
Validation auc = 0.682160
Epoch 60
Validation auc = 0.653991
Epoch 61
Validation auc = 0.562911
Epoch 62
Validation auc = 0.634507
Epoch 63
Validation auc = 0.666197
Epoch 64
Loss = 8.8978e+00, PNorm = 70.2483, GNorm = 239.8463, lr_0 = 9.0754e-04
Validation auc = 0.683099
Epoch 65
Validation auc = 0.682394
Epoch 66
Validation auc = 0.679343
Epoch 67
Validation auc = 0.654930
Epoch 68
Validation auc = 0.651878
Epoch 69
Loss = 1.0968e+01, PNorm = 70.6415, GNorm = 983.4081, lr_0 = 8.9978e-04
Validation auc = 0.624883
Epoch 70
Validation auc = 0.719249
Epoch 71
Validation auc = 0.676995
Epoch 72
Validation auc = 0.698826
Epoch 73
Validation auc = 0.669953
Epoch 74
Loss = 6.3768e+00, PNorm = 71.0945, GNorm = 298.2621, lr_0 = 8.9209e-04
Validation auc = 0.693897
Epoch 75
Validation auc = 0.718779
Epoch 76
Validation auc = 0.690141
Epoch 77
Validation auc = 0.552113
Epoch 78
Validation auc = 0.625352
Epoch 79
Loss = 2.8762e+01, PNorm = 71.4088, GNorm = 1752.0485, lr_0 = 8.8447e-04
Validation auc = 0.705869
Epoch 80
Validation auc = 0.657277
Epoch 81
Validation auc = 0.723005
Epoch 82
Validation auc = 0.721596
Epoch 83
Validation auc = 0.689906
Epoch 84
Loss = 1.1180e+01, PNorm = 71.6922, GNorm = 247.3318, lr_0 = 8.7691e-04
Validation auc = 0.701878
Epoch 85
Validation auc = 0.707512
Epoch 86
Validation auc = 0.690610
Epoch 87
Validation auc = 0.667840
Epoch 88
Validation auc = 0.650235
Epoch 89
Loss = 3.2428e+00, PNorm = 71.8904, GNorm = 416.3010, lr_0 = 8.6941e-04
Validation auc = 0.653991
Epoch 90
Validation auc = 0.631221
Epoch 91
Validation auc = 0.670423
Epoch 92
Validation auc = 0.674178
Epoch 93
Validation auc = 0.662676
Epoch 94
Loss = 2.5934e+00, PNorm = 72.1269, GNorm = 394.6825, lr_0 = 8.6198e-04
Validation auc = 0.714554
Epoch 95
Validation auc = 0.664085
Epoch 96
Validation auc = 0.713615
Epoch 97
Validation auc = 0.711502
Epoch 98
Validation auc = 0.683333
Epoch 99
Loss = 5.4184e+00, PNorm = 72.4331, GNorm = 374.6513, lr_0 = 8.5461e-04
Validation auc = 0.720657
Epoch 100
Validation auc = 0.676761
Epoch 101
Validation auc = 0.686150
Epoch 102
Validation auc = 0.683099
Epoch 103
Validation auc = 0.677465
Epoch 104
Loss = 2.5706e+00, PNorm = 72.7703, GNorm = 586.6066, lr_0 = 8.4730e-04
Validation auc = 0.686385
Epoch 105
Validation auc = 0.674413
Epoch 106
Validation auc = 0.635211
Epoch 107
Validation auc = 0.700469
Epoch 108
Validation auc = 0.670657
Epoch 109
Loss = 7.5137e+00, PNorm = 73.0920, GNorm = 669.5068, lr_0 = 8.4006e-04
Validation auc = 0.676291
Epoch 110
Validation auc = 0.693662
Epoch 111
Validation auc = 0.729108
Epoch 112
Validation auc = 0.695540
Epoch 113
Validation auc = 0.738498
Epoch 114
Loss = 5.4399e+00, PNorm = 73.3173, GNorm = 185.2471, lr_0 = 8.3288e-04
Validation auc = 0.673239
Epoch 115
Validation auc = 0.719249
Epoch 116
Validation auc = 0.660094
Epoch 117
Validation auc = 0.660563
Epoch 118
Validation auc = 0.699296
Epoch 119
Loss = 3.6959e+00, PNorm = 73.6302, GNorm = 699.3802, lr_0 = 8.2576e-04
Validation auc = 0.642488
Epoch 120
Validation auc = 0.640376
Epoch 121
Validation auc = 0.685211
Epoch 122
Validation auc = 0.671831
Epoch 123
Validation auc = 0.681925
Epoch 124
Loss = 3.5045e+00, PNorm = 73.9275, GNorm = 424.2645, lr_0 = 8.1870e-04
Validation auc = 0.666901
Epoch 125
Validation auc = 0.696009
Epoch 126
Validation auc = 0.692488
Epoch 127
Validation auc = 0.650235
Epoch 128
Validation auc = 0.692958
Epoch 129
Loss = 5.1181e+00, PNorm = 74.2215, GNorm = 784.3635, lr_0 = 8.1170e-04
Validation auc = 0.720188
Epoch 130
Validation auc = 0.675352
Epoch 131
Validation auc = 0.733333
Epoch 132
Validation auc = 0.724178
Epoch 133
Validation auc = 0.653756
Epoch 134
Loss = 8.0033e+00, PNorm = 74.4851, GNorm = 1059.1745, lr_0 = 8.0476e-04
Validation auc = 0.767840
Epoch 135
Validation auc = 0.703756
Epoch 136
Validation auc = 0.775352
Epoch 137
Validation auc = 0.711972
Epoch 138
Validation auc = 0.782864
Epoch 139
Loss = 4.3309e+00, PNorm = 74.7626, GNorm = 247.9575, lr_0 = 7.9788e-04
Validation auc = 0.729577
Epoch 140
Validation auc = 0.761033
Epoch 141
Validation auc = 0.724648
Epoch 142
Validation auc = 0.725352
Epoch 143
Validation auc = 0.649061
Epoch 144
Loss = 3.9219e+00, PNorm = 74.9310, GNorm = 351.9203, lr_0 = 7.9106e-04
Validation auc = 0.710563
Epoch 145
Validation auc = 0.651643
Epoch 146
Validation auc = 0.648826
Epoch 147
Validation auc = 0.684977
Epoch 148
Validation auc = 0.699531
Epoch 149
Loss = 4.6597e+00, PNorm = 75.2467, GNorm = 117.9348, lr_0 = 7.8430e-04
Validation auc = 0.709155
Epoch 150
Validation auc = 0.727465
Epoch 151
Validation auc = 0.698357
Epoch 152
Validation auc = 0.734507
Epoch 153
Validation auc = 0.707512
Epoch 154
Loss = 2.6699e+00, PNorm = 75.4971, GNorm = 572.1487, lr_0 = 7.7759e-04
Validation auc = 0.695775
Epoch 155
Validation auc = 0.720892
Epoch 156
Validation auc = 0.715493
Epoch 157
Validation auc = 0.688028
Epoch 158
Validation auc = 0.717136
Epoch 159
Loss = 2.0812e+00, PNorm = 75.8409, GNorm = 204.9286, lr_0 = 7.7095e-04
Validation auc = 0.716432
Epoch 160
Validation auc = 0.681690
Epoch 161
Validation auc = 0.731925
Epoch 162
Validation auc = 0.722300
Epoch 163
Validation auc = 0.649061
Epoch 164
Loss = 8.9519e+00, PNorm = 76.0706, GNorm = 952.6145, lr_0 = 7.6436e-04
Validation auc = 0.735446
Epoch 165
Validation auc = 0.668779
Epoch 166
Validation auc = 0.684038
Epoch 167
Validation auc = 0.694366
Epoch 168
Validation auc = 0.749296
Epoch 169
Loss = 5.8955e+00, PNorm = 76.3954, GNorm = 192.5576, lr_0 = 7.5782e-04
Validation auc = 0.707981
Epoch 170
Validation auc = 0.685915
Epoch 171
Validation auc = 0.688967
Epoch 172
Validation auc = 0.699765
Epoch 173
Validation auc = 0.697183
Epoch 174
Loss = 2.3018e+00, PNorm = 76.6319, GNorm = 273.2598, lr_0 = 7.5134e-04
Validation auc = 0.673239
Epoch 175
Validation auc = 0.711033
Epoch 176
Validation auc = 0.683099
Epoch 177
Validation auc = 0.637324
Epoch 178
Validation auc = 0.652347
Epoch 179
Loss = 2.9931e+00, PNorm = 76.9253, GNorm = 389.3245, lr_0 = 7.4492e-04
Validation auc = 0.752113
Epoch 180
Validation auc = 0.661268
Epoch 181
Validation auc = 0.754695
Epoch 182
Validation auc = 0.750235
Epoch 183
Validation auc = 0.686150
Epoch 184
Loss = 5.1029e+00, PNorm = 77.2896, GNorm = 859.8407, lr_0 = 7.3855e-04
Validation auc = 0.728638
Epoch 185
Validation auc = 0.768779
Epoch 186
Validation auc = 0.776056
Epoch 187
Validation auc = 0.725822
Epoch 188
Validation auc = 0.755164
Epoch 189
Loss = 9.4172e-01, PNorm = 77.6441, GNorm = 67.9817, lr_0 = 7.3224e-04
Validation auc = 0.737324
Epoch 190
Validation auc = 0.766197
Epoch 191
Validation auc = 0.704460
Epoch 192
Validation auc = 0.745540
Epoch 193
Validation auc = 0.734977
Epoch 194
Loss = 4.3667e-01, PNorm = 77.9818, GNorm = 365.2419, lr_0 = 7.2598e-04
Validation auc = 0.699765
Epoch 195
Validation auc = 0.704460
Epoch 196
Validation auc = 0.719953
Epoch 197
Validation auc = 0.662911
Epoch 198
Validation auc = 0.680516
Epoch 199
Loss = 3.3342e+00, PNorm = 78.3745, GNorm = 552.6106, lr_0 = 7.1977e-04
Validation auc = 0.708451
Epoch 200
Validation auc = 0.684977
Epoch 201
Validation auc = 0.671596
Epoch 202
Validation auc = 0.653052
Epoch 203
Validation auc = 0.698122
Epoch 204
Loss = 5.0177e+00, PNorm = 78.9258, GNorm = 65.7267, lr_0 = 7.1362e-04
Validation auc = 0.622535
Epoch 205
Validation auc = 0.706338
Epoch 206
Validation auc = 0.665023
Epoch 207
Validation auc = 0.744366
Epoch 208
Validation auc = 0.689202
Epoch 209
Loss = 8.5017e+00, PNorm = 79.3899, GNorm = 520.2982, lr_0 = 7.0752e-04
Validation auc = 0.739202
Epoch 210
Validation auc = 0.745775
Epoch 211
Validation auc = 0.761268
Epoch 212
Validation auc = 0.689202
Epoch 213
Validation auc = 0.719484
Epoch 214
Loss = 2.2447e+00, PNorm = 79.8027, GNorm = 197.8442, lr_0 = 7.0147e-04
Validation auc = 0.749061
Epoch 215
Validation auc = 0.735915
Epoch 216
Validation auc = 0.667371
Epoch 217
Validation auc = 0.701174
Epoch 218
Validation auc = 0.698122
Epoch 219
Loss = 1.2964e+00, PNorm = 80.1159, GNorm = 114.9706, lr_0 = 6.9548e-04
Validation auc = 0.674178
Epoch 220
Validation auc = 0.739906
Epoch 221
Validation auc = 0.724648
Epoch 222
Validation auc = 0.686150
Epoch 223
Validation auc = 0.720423
Epoch 224
Loss = 2.3495e+00, PNorm = 80.3739, GNorm = 237.8774, lr_0 = 6.8953e-04
Validation auc = 0.725822
Epoch 225
Validation auc = 0.723709
Epoch 226
Validation auc = 0.692254
Epoch 227
Validation auc = 0.716432
Epoch 228
Validation auc = 0.715962
Epoch 229
Loss = 1.1415e+00, PNorm = 80.6490, GNorm = 156.4955, lr_0 = 6.8364e-04
Validation auc = 0.697887
Epoch 230
Validation auc = 0.647418
Epoch 231
Validation auc = 0.641315
Epoch 232
Validation auc = 0.679108
Epoch 233
Validation auc = 0.682394
Epoch 234
Loss = 8.5532e-01, PNorm = 80.8440, GNorm = 330.1739, lr_0 = 6.7779e-04
Validation auc = 0.675117
Epoch 235
Validation auc = 0.672300
Epoch 236
Validation auc = 0.677230
Epoch 237
Validation auc = 0.692254
Epoch 238
Validation auc = 0.687559
Epoch 239
Loss = 2.2025e+00, PNorm = 81.0249, GNorm = 383.3831, lr_0 = 6.7200e-04
Validation auc = 0.666197
Epoch 240
Validation auc = 0.689202
Epoch 241
Validation auc = 0.680986
Epoch 242
Validation auc = 0.692488
Epoch 243
Validation auc = 0.688967
Epoch 244
Loss = 1.2354e+00, PNorm = 81.2634, GNorm = 91.9384, lr_0 = 6.6625e-04
Validation auc = 0.699531
Epoch 245
Validation auc = 0.688498
Epoch 246
Validation auc = 0.669014
Epoch 247
Validation auc = 0.676526
Epoch 248
Validation auc = 0.677230
Epoch 249
Loss = 1.4543e+00, PNorm = 81.3849, GNorm = 72.0881, lr_0 = 6.6056e-04
Validation auc = 0.661502
Epoch 250
Validation auc = 0.661972
Epoch 251
Validation auc = 0.696244
Epoch 252
Validation auc = 0.687089
Epoch 253
Validation auc = 0.660563
Epoch 254
Loss = 2.5029e+00, PNorm = 81.6060, GNorm = 386.6677, lr_0 = 6.5491e-04
Validation auc = 0.675587
Epoch 255
Validation auc = 0.702817
Epoch 256
Validation auc = 0.665962
Epoch 257
Validation auc = 0.672066
Epoch 258
Validation auc = 0.690845
Epoch 259
Loss = 1.7242e+00, PNorm = 81.9582, GNorm = 185.4993, lr_0 = 6.4931e-04
Validation auc = 0.692488
Epoch 260
Validation auc = 0.663380
Epoch 261
Validation auc = 0.656808
Epoch 262
Validation auc = 0.657746
Epoch 263
Validation auc = 0.702347
Epoch 264
Loss = 1.7804e+00, PNorm = 82.3399, GNorm = 125.5599, lr_0 = 6.4376e-04
Validation auc = 0.716901
Epoch 265
Validation auc = 0.643662
Epoch 266
Validation auc = 0.652582
Epoch 267
Validation auc = 0.713146
Epoch 268
Validation auc = 0.702113
Epoch 269
Loss = 2.9076e+00, PNorm = 82.6363, GNorm = 154.3197, lr_0 = 6.3826e-04
Validation auc = 0.623239
Epoch 270
Validation auc = 0.653521
Epoch 271
Validation auc = 0.674178
Epoch 272
Validation auc = 0.636620
Epoch 273
Validation auc = 0.642019
Epoch 274
Loss = 1.5188e+00, PNorm = 83.0611, GNorm = 404.0327, lr_0 = 6.3280e-04
Validation auc = 0.653286
Epoch 275
Validation auc = 0.618545
Epoch 276
Validation auc = 0.613146
Epoch 277
Validation auc = 0.653756
Epoch 278
Validation auc = 0.650235
Epoch 279
Loss = 1.3826e+00, PNorm = 83.4979, GNorm = 131.5636, lr_0 = 6.2739e-04
Validation auc = 0.637793
Epoch 280
Validation auc = 0.662207
Epoch 281
Validation auc = 0.659390
Epoch 282
Validation auc = 0.653052
Epoch 283
Validation auc = 0.586150
Epoch 284
Loss = 3.2534e+00, PNorm = 83.9584, GNorm = 743.5774, lr_0 = 6.2203e-04
Validation auc = 0.646009
Epoch 285
Validation auc = 0.691549
Epoch 286
Validation auc = 0.667136
Epoch 287
Validation auc = 0.625587
Epoch 288
Validation auc = 0.689437
Epoch 289
Loss = 1.9244e+00, PNorm = 84.2941, GNorm = 480.3035, lr_0 = 6.1671e-04
Validation auc = 0.694836
Epoch 290
Validation auc = 0.680282
Epoch 291
Validation auc = 0.684507
Epoch 292
Validation auc = 0.665258
Epoch 293
Validation auc = 0.671362
Epoch 294
Loss = 8.3340e-01, PNorm = 84.6072, GNorm = 96.9070, lr_0 = 6.1144e-04
Validation auc = 0.658685
Epoch 295
Validation auc = 0.661737
Epoch 296
Validation auc = 0.646479
Epoch 297
Validation auc = 0.665023
Epoch 298
Validation auc = 0.668310
Epoch 299
Loss = 1.5899e+00, PNorm = 84.8826, GNorm = 370.5279, lr_0 = 6.0621e-04
Validation auc = 0.664554
Model 0 best validation auc = 0.786620 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.801927
Ensemble test auc = 0.801927
10-fold cross validation
	Seed 0 ==> test auc = 0.829922
	Seed 1 ==> test auc = 0.719446
	Seed 2 ==> test auc = 0.815172
	Seed 3 ==> test auc = 0.805539
	Seed 4 ==> test auc = 0.727875
	Seed 5 ==> test auc = 0.712523
	Seed 6 ==> test auc = 0.831427
	Seed 7 ==> test auc = 0.752559
	Seed 8 ==> test auc = 0.812161
	Seed 9 ==> test auc = 0.801927
Overall test auc = 0.780855 +/- 0.044971
Elapsed time = 0:04:56
